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Overview 


yy With the COVID-19 pandemic disrupting 
education across the globe in early 2020, 
many educators found themselves rapidly 
transitioning to remote learning without the 
appropriate knowledge, skills or resources. 
For many K-12 students, this resulted in less 
than optimal learning experiences. 


To ensure that high quality online learning is being 
offered in the future, the Kingdom of Saudi Arabia 
commissioned a study to understand the state of online 
learning pre-, peri- and post-COVID based on the 
academic period from March 9-May 14, 2020. The 
intention was to determine opportunities for 
improvement while identifying areas of excellence that 
could be highlighted against an evaluation framework 
for quality online learning. Using this information, K-12 
programs within the Kingdom of Saudi Arabia would be 
positioned to make improvements to the overall student 


learning experience. 
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Executive Summary 


To ensure that high quality online learning is being offered in the future, the Kingdom 
of Saudi Arabia commissioned a study to understand the state of online K-12 education 
pre-, peri- and post-COVID. The intention was to determine opportunities for 
improvement while identifying areas of excellence that could be highlighted against 
an evaluation framework for quality K-12 online learning. Using this information, K-12 
education within the Kingdom of Saudi Arabia would be positioned to make 
improvements to the overall student learning experience. 


The study focused on eight dimensions, each with several subdimensions, and included survey and interview 
data, contextualized within the KSA K-12 environment for both development and analysis. Dimensions included 
Leadership, Curriculum Design and Planning, Online Teaching and Learning, Assessment, Technology, Student 
Support, Training and Support, and Evaluation and Continuous Improvement. Each of these dimensions and their 
associated subdimensions are discussed at length in the report, but there were several overarching themes that 
emerged. 


Themes 


Clear themes and trends, which can be best understood in the context of the rapid shift to emergency remote 
teaching (ERT) following the pandemic, emerged throughout the study. 


Prior to COVID, online education supporting K-12 in the Kingdom of Saudi Arabia was largely supplementary to 
traditional instruction, though increasing infrastructure, capacity, and capability was a priority as part of Vision 
2030. The pandemic forced a rapid shift in the delivery of K-12 education, including significantly increased 
demands on schools and infrastructure to support online learning. The study results illuminated areas of 
preparedness as well as opportunities for improvement to support large-scale online education for K-12 moving 
forward. 


Following COVID, several dimensions and subdimensions showed clear signs of improvement in responses to both 
survey and interview data provided by administrators, staff, teachers, and students. This could be an indication 
that communications regarding these areas increased or were more clear and understood, and/or that priority 
was placed on these items to enable as successful a transition to online learning as possible. Of particular note, 
though opportunities for further improvement are indicated and provided in the report, the following items 
resulted in increases in positive perception post-COVID. 

e TheLeadership dimension results indicated that post-COVID, stakeholders’ perceptions of and 
experiences with governance and strategies related to online learning were more positive. 
Communications and opportunities for stakeholder involvement appear to have effectively increased in 
the management of the pandemic. Other measures in this dimension (i.e. policies, processes, and both 


fiscal and human resources) indicate increased positivity regarding implementation of leadership 
initiatives for online learning that suggest a positive trajectory, though they may have been unevenly 
implemented due to circumstances created by and/or pre-existing to the pandemic. 

e COVID-19 presented a pressing opportunity for teachers to evaluate the communication and 
implementation of learning objectives, review content and materials including OER for currency and 
engagement format, and to enhance overall communication, engagement, and course interaction ina 
digital environment. The study suggests that their immediate actions resulted in increased success in 
these areas, and that they are well-positioned for continual evaluation and improvement moving 
forward. 

e Student support is critical in the educational environment, and moreso during the COVID environment, 
when needs may have changed quickly and prior methods of providing student support are no longer 
available. Study results indicate that stakeholders are aware of these needs and have taken steps to both 
evaluate needs and provide remote support. Ongoing efforts to both assess and address student support 
needs and solutions are evident, and continued efforts will further allow schools to provide necessary 
resources. 


These areas indicate preparedness for online education, though further developments may be made through the 
recommendations in this report. In addition, there were some areas that require attention based on study results. 
e The pandemic likely revealed important pedagogical considerations that may not have been originally 
included in the curriculum as well as the social-emotional needs that learners may have. Key areas of 
focus include: 

o Ensuring that instructional design follows quality standards and presents equitable 
opportunities for all students. 

o Ensuring that course elements such as alignment of outcomes/objectives with materials, content 
and assessment as well as expectations and provision of feedback are provided in a consistent 
and effective manner, and that academic support is available to and accessed by students. 

o Reviewing and implementing solutions to address issues of internet access, reliability, coverage, 
and operability. Each of these issues was included in online education needs prior to COVID, but 
challenges were exacerbated and immediate needs accelerated in the wake of the pandemic. 
Attention to this is critical for continued success in online education. 


Emerging Needs 


Survey and interview analyses resulted in recommendations across all eight dimensions that will allow for the 
development and implementation of higher capacity and quality in online learning. Themes throughout, and 
particularly related to communication and technology indicate that though some challenges existed prior to 
COVID-19, the rapid shift to online learning following the pandemic resulted in increased visibility of challenges 
(both prior to and following COVID-19) as well as increased need in these areas. Study results indicate that 
immediate action was taken on these items, though challenges remain. Moving forward, building on the pre- and 
peri-COVID solutions as well as directing attention to areas critical for the provision of effective online education is 
essential. 
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Scenarios 


The two scenarios presented from this data include online teaching and learning goals outside of consideration 
for COVID, and implementing effective online teaching and learning as influenced by COVID. 


Recommendations provided encompass both scenarios, although some present long-term strategies for 
increasing the capacity and effectiveness of online teaching and learning (such as long-range strategic planning 
and development of processes for evaluation and continuous improvement). Others, while no less important for 
long-term solutions, are a higher priority to manage effective learning experiences as well as support needs during 
COVID (for example, ensuring that technology is sufficient to support high-volume online courses and providing 
clear expectations for communication, assessment, and interactions in the online environment). All 
recommendations are discussed in detail in the report, and summarized (including identification of COVID 
priorities) in Appendix IV: Future Action Framework. 


Implications 


Prior to COVID-19 and the commissioning of this study, the Kingdom of Saudi Arabia included online teaching and 
learning as a need and goal of Vision 2030, and invested in the development and success of online education as a 
key component of attaining long-term K-12 goals. These investments included a variety of offerings (online 
platform, OER, e-Content, YouTube, satellite television) and continuous improvement efforts, and resulted in a 
quick response to the pandemic and immediate shift to remote instruction. This study illuminated overall needs 
to enhance country-wide solutions related to accomplishing these goals. The pandemic required rapid 
acceleration in the development and delivery of fully online courses for a high volume of students, and in addition 
to existing needs, this study further illuminated critical needs to support success. 


The recommendations resulting from this study are designed to support both long-term and immediate needs to 
develop and deliver effective online teaching and learning in both scenarios. While many elements necessary to 
support online education were evident pre-COVID, and clear efforts have been made to provide effective online 
learning during the pandemic, implications resulting from this study indicate that there are many actions that may 
be implemented to better support Vision 2030 goals and address needs that emerged with COVID-19 and persist 
throughout the shift resulting from the pandemic. 


Introduction 


Traditional education for K-12 educators primarily focuses on how to teach students 
in a face-to-face environment. In late 2019 and early 2020, the world became aware of 
a new threat to society, COVID-19. Over the course of a few months, this disease 
spread around the world until it was deemed a pandemic. This forced many K-12 
educators into unfamiliar territory where they needed to quickly shift from teaching 
students in the classroom to a remote experience. Too few were adequately prepared 
to teach in this modality since professional development for teachers has typically 
revolved around the needs of teaching in a classroom. 


To ensure that high quality online learning is being offered in the future, the Kingdom of Saudi Arabia 
commissioned a study to understand the state of online K-12 education pre-, peri- and post-COVID. The intention 
was to determine opportunities for improvement while identifying areas of excellence that could be highlighted 
against an evaluation framework for quality K-12 online learning. Using this information, K-12 education within 
the Kingdom of Saudi Arabia would be positioned to make improvements to the overall student learning 
experience. 


Given their vast network and expertise, the Online Learning Consortium (OLC) was asked to lead this project with 
the assistance of several key partners from around the world. This included experts from ISTE, UNESCO IITE, 
Quality Matters, the University of Wisconsin-Milwaukee Center for Distance Education and Technological 
Advancement (DETA), Michigan Virtual, and the Illinois Virtual School. Partners were included at different stages of 
the study but primarily provided feedback and input into the analysis and recommendations provided within this 
report. 


This report has been organized into four primary sections as well as several appendices. The introduction provides 
context for the study, the methodology section provides information regarding the development and analysis of 
both quantitative survey instrumentation and qualitative protocols, and includes a summary of participants, the 
recommendations section provides data interpretation and recommendations by dimension and subdimension 
(aligned with the Future Action Framework), and the considerations for future study section contains 
recommendations for further research based on this study as well as findings indicating potential future 
expansion. Finally, the appendices provide the Future Action Framework summary of recommendations, the 
literature review, survey and interview protocols, and a list of figures. 
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Methodology 


The K-12 study began with the development of an eight-dimension framework, 
designed by the National eLearning Center (NELC) and collaboratively finalized by 
NELC and OLC. The framework, including dimensions and subdimensions, is attached 
in Appendix I: NELC Evaluation Framework, and was used to inform both survey and 
interview protocols. 


The Evaluation Framework was used to design Appendix II: Research Methodology Framework Mapping as well as 
the reported Research Findings. Appendix IV: Future Action Framework, is the culmination of the evaluation and 
research frameworks with recommendations summarized as described later in this section. 


Drawing on the eight dimensions and subdimensions, and informed by policy documents provided by NELC, 
descriptive surveys were developed for K-12 administrators, staff, teachers, parents, and students by DETA and 
OLC. The purpose of the surveys was to understand the national position to provide quality online learning across 
institutions during the COVID-19 pandemic in the Kingdom of Saudi Arabia (KSA) and to identify areas for future 
investment and improvement. The recommendations resulting from survey responses provide information to 
support capacity and quality in online education throughout the country to enhance the potential for Vision 2030 
excellence in a diversified and knowledge-based economy. Survey instrumentation was developed based ona 
scan of other instrumentation and tools, including national and international products, literature, and recent 
research relevant to the dimensions and subdimensions. Survey items were developed to measure the 
participants’ attitudes, opinions, and beliefs about online learning at their institutions, to assess the performance 
of the institutions and different functions of the institutions to support online learning and quality, and to identify 
areas of challenge and opportunities for improvement in future academic terms. All survey items are considered 
to have both construct and content validity. 


The survey was delivered through Survey Monkey and was designed as one survey with branching logic to ease 
data collection and data set management. Items were individually coded by stakeholder group and dimension for 
analysis and storage. The survey was sent to educational offices throughout the country, and they provided it to 
school principals, who distributed the request to staff, teachers, parents and students. 


Survey analysis included descriptive statistical analysis completed in SPSS, disaggregated by stakeholder group 
(administrators, staff, teachers, parents, and students) as well as by each dimension. 


There were 318,243 completed surveys. Participants included 24,809 (7.8%) administrators, 17,163 (5.4%) staff 
members, 108,505 (34.1%) teachers, 112,891 (35.5%) parents, and 54,875 (17.2%) students. Please note that a 
number of respondents chose not to answer demographic questions beyond their role, and the figures described 
below represent those that did provide responses. 


Administrator, Staff, and Teacher Participants 


Respondents from this stakeholder group represented 13 regions in the Kingdom of Saudi Arabia, and included 
governmental, national, and international school representation. There were 43,375 (72.9%) governmental, 12,612 
(21.7%) national, and 3,175 (5.5%) international school participants. The majority of administrator, staff, and 
teacher participants, 50,912 (87.1%) indicated Saudi nationality with 7,520 (12.9%) identifying as non-Saudi. There 
were 23,672 (40.6%) male and 34,690 (59.4%) female respondents. Respondents represented a range of 
experience in education, elearning, and time at their current schools (Table 1) as well as a diversity of 
specializations (Table 2). 


Table 1: Administrator, Staff, and Teacher Years of Experience 


a Years of Education Experience | Years of eLearning Experience | Years at Current School 
0-5 years 9,830 17.6% 29,692 65.3% 26,877 48.9% 
6-10 years 12,781 22.9% 8,321 18.3% 14,834 27.0% 
11-15 years | 9,067 16.3% 3,681 5,466 9.9% 


21-25 years | 8,652 15.5% 933 2,365 4.3% 
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Table 2: Teacher Specialization 


Administration Science 
Arabic Language 
Art Education 
Biology 
Chemistry 
Chinese Language 
Computers 

Earth Science 
English Language 
Family Education 
Islamic Education 


Kindergarten 


Mathematics 

Physical Education 

Physics 

Psychosocial Skills 

Science 

Social and National Education 
Special Education 


Student Counseling 


10 


209 


176 


4,219 
1,550 


7,145 


1,555 


5,752 


747 


387 


959 


378 


18.7% 
2.9% 
3.6% 
3.2% 
0.1% 
3.5% 
0.4% 
9.6% 
3.5% 
16.3% 
3.6% 
0.6% 
13.1% 
1.7% 
2.8% 
0.9% 
5.8% 
6.3% 
2.2% 


0.9% 


Parent Participants 


Parent respondents represented 13 regions in Saudi Arabia, and included 23,799 (71.6%) whose children attended 
governmental, 6,919 (20.8%) national, and 2,543 (7.6%) international schools. The majority of parents, 28,364 
(84.8%) indicated Saudi nationality, with 5,082 (15.2%) identifying as non-Saudi. Participants included 13,996 
(42.0%) male and 19,355 (58.0%) female parents with a range of educational levels (Table 3). 


Table 3: Parent Education Levels 


Student Participants 


Student respondents represented 13 regions of Saudi Arabia, with 17,705 (81.3%) attending government, 2,876 
(13.2%) national, and 1,201 (5.5%) international schools. The majority of students, 17,477 (80.1%) identified as 
Saudi, with 4,347 (19.9%) indicating non-Saudi nationality. There were 5,369 (24.6%) male and 16,455 (75.4%) 
female student respondents, representing a range of grade levels (Table 4). It should be noted that the grade level 
distribution represented in this survey indicates a limitation in the interpretation of student responses below the 
elementary level, as these participants make up a small percentage of the overall student sample. 
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Table 4: Student Grade Levels 


KG1 


KG2 


KG3 


Following survey development, several dimensions were identified as valuable for gathering more in-depth and 
nuanced information, and interview protocols were developed for administrators, staff, teachers, and parents. 
Interviews were semi-structured, with each protocol containing background information, a number of questions 
organized by dimension, and with opportunities for participants to provide detailed information as well as any 
additional comments. 


NELC staff selected participants and conducted interviews via Zoom or telephone calls. Interview responses were 
provided in text, and uploaded to a shared drive. Interviews were then coded using case classifications to identify 
participant characteristics and gather baseline data on interview questions. In-text coding was completed by 
dimension and subdimension to provide detailed information about participants’ attitudes, opinions, beliefs, and 
experiences with interview questions. 
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Interview participants included 30 administrators/staff, 30 teachers, and 9 parents. 


Quantitative and qualitative results were analyzed to develop recommendations aligned to the Future Action 
Framework and organized by dimension and subdimension. Recommendations are described below, and are 
summarized in Appendix IV: Future Action Framework. These recommendations include narrative, target 
audience, needs, initiatives, goals, action lines, critical success factors (to be written as “CSFs” in all following 
tables), and key performance indicators (to be written as “KPIs” in all following tables). 


Recommendations 


The K-12 study resulted in description of both survey and interview findings, 
including a number of recommendations. 


Recommendations are organized by dimension, sub-dimension, and target audience, and include all relevant 
factors including needs, initiatives, goals, actions, critical success factors, and key performance indicators. 
Recommendations are described below, and are summarized in Appendix IV: Future Action Framework. 


Dimension |; Leadership 


The Leadership dimension measures six subdimensions including: Governance, Strategies, Policies, Processes, 
Resource Allocation, and Staffing/Innovation. 


Leadership readiness appears to have been commensurate with the pace of online learning development and 
delivery in a pre-COVID environment. Post-COVID, survey and interview results indicate that although leadership 
related to online learning was not fully prepared for the shift due to the pandemic, foundational elements of each 
subdimension were sufficient to provide positive direction. Areas most in need of readiness improvement 
included the alignment and inclusion of online learning in strategic planning, more even implementation of 
policies and processes, and adequate resources and staffing to support a fully-online environment. 


Within the surveys, this dimension was examined from the perspective of administrators, staff and teachers. 
Students and parents were not asked to rate items from these subdimensions since it would be unlikely that they 
would have appropriate insight into these areas. 


Through the interviews that were conducted, more depth of information was acquired regarding the school’s 
strategy around online learning as well as the sufficiency of resources to adequately support and sustain online 
learning. Within the interviews for the Leadership dimension, it was noted that there is a wide variance in the level 
of preparedness found at the institutions within the Kingdom of Saudi Arabia. A definite difference was noted 
regarding the needs identified by administrators, staff and teachers. While not identified directly through the 
interview questions, these differences may be due in part to communications and level of involvement in online 
activities by the various parties. 
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Overall findings indicate strong agreement from administrators, staff, and teachers alike that leadership 
subdimensions of governance, strategy, and policy are in place to support quality online learning throughout the 
Kingdom. These findings are further supported by the OECD report How the COVID-19 Pandemic is Changing 
Education: A Perspective from Saudi Arabia, in their finding that schools were well-positioned to manage the 
pandemic, though internet access was a concern. With respect to processes, resource allocation, and innovation, 
there is also broad agreement from these stakeholders, yet there is also opportunity to more equitably allocate 
resources, pursue process improvement, and open innovation opportunities. A review of the data and 
recommendations for each of these subdimensions follows. 


Subdimension: Governance 


A review of the survey findings for the governance subdimension reflect that the majority of survey participants 
agreed or strongly agreed that their institution has the appropriate governance structure in place to support 
online education. Across the three questions for this subdimension, all three role groups -- administrators, staff, 
and teachers -- indicated an increase post-COVID related to leadership governance. Interviews with 
administrators, staff, and teachers did not yield particular insights that suggested a departure from the survey 
data, outside of a desire to receive more timely communication about decisions related to online learning ina 
fluctuating context of COVID-19 and the rapid shifts it can create for school-level actors. 


With respect to leadership governance structures that enable “clear, effective, and comprehensive decision 
making related to online education,” survey respondents indicated a pre-COVID satisfaction of 63.9%. This 
number increased by nearly 7% to 70.5% across all three groups post-COVID, with a consistent distribution of 
satisfaction within role groups (Figure 1). 
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THE GOVERNANCE STRUCTURE ENABLES CLEAR, EFFECTIVE, 
AND COMPREHENSIVE DECISION MAKING RELATED TO ONLINE 
EDUCATION. 


mAdministrators mStaff mTeachers 


A STRONGLY AGREE 
> AGREE fi 52.1% EE WAS 52.3% 
id NEUTRAL 
- DISAGREE. |MOESACES ARS 
< STRONGLY DISAGREE 
STRONGLY AGREE _ | MSSSGR SCAR 
= cree a a ee 
c NEUTRAL 
7 DISAGREE | BRREYAREyA/R Ges 
_ STRONGLY DISAGREE | CBEXAYAUZK0Y5 
Figure 1 


Similar to responses about leadership governance providing clear decision making, a preponderance of survey 
respondents agreed that “the governance structure incorporates a diverse body of decision-makers, including 
teachers and staff, and those who are knowledgeable about K-12 online learning.” Pre-COVID, the agreement 
across all three groups was 64.1%, and jumped to 67.7% post-COVID. Again, all three role groups had relatively 
consistent responses across the agreement ranges (Figure 2). 


THE GOVERNANCE STRUCTURE INCORPORATES A DIVERSE 
BODY OF DECISION-MAKERS, INCLUDING TEACHERS AND 
STAFF, AND THOSE WHO ARE KNOWLEDGEABLE ABOUT K-12 
ONLINE LEARNING. 


mAdministrators mStaff mTeachers 


a STRONGLY AGREE 
> AGREE f 50.9% 53.9% 51.1% 
4 NEUTRAL | QRSKOSAREEN ICH 0) RCRD 
2 DISAGREE 
< STRONGLY DISAGREE 
STRONGLY AGREE (@RSSAmEDX AREY 
= AGREE 50.2% 53.6% 50.2% 
G NEUTRAL 
uw DISAGREE 
“STRONGLY DISAGREE |MMDLOREIIENS 
Figure 2 
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Related to incorporating a diverse body of knowledgeable decision-makers, the minority of teacher respondents 
who disagreed or strongly disagreed decreased from pre-COVID to post-COVID, from 18.7% to 15.8%. The number 
of administrators also decreased from 21.0% to 18.1%. Broadly, this decrease in disagreement indicates that the 
transition through the COVID pandemic showed both administrators and teachers that the governance structure is 
incorporating a more diverse body of knowledgeable stakeholders in decision-making. 


In line with the first two questions on leadership governance, a majority of respondents agreed or strongly agreed 
that current governance structures “requires and enables systematic and continuous improvement related to 
online education.” While the three groups had a total agreement response of 69.7% pre-COVID, this question saw 
an increase in agreement with the teaching subgroup post-COVID, with overall satisfaction at 73.7%. 
Commensurate with the other questions in this subdimension, the responses across role groups were consistent 
(Figure 3). 


THE GOVERNANCE STRUCTURE REQUIRES AND ENABLES 
SYSTEMATIC AND CONTINUOUS IMPROVEMENT RELATED TO 
ONLINE EDUCATION. 


mAdministrators mStaff mTeachers 


STRONGLY AGREE 18.9% 17.3% 18.6% 
S AGREE 54.8% 55.9% 55.1% 
= DISAGREE 
: STRONGLY DISAGREE PEW. 

STRONGLY AGREE 16.1% 14.5% 15.4% 
= AGREE 53.4% 55.9% 54.3% 
o NEUTRAL _ | RESUSeZ RES y/ i SEZ 
e DISAGREE | HOMRZSNeaieneys 

STRONGLY DISAGREE 736144694 
Figure 3 


Expert Recommendation: Share and communicate decision-making rationale and 
processes with local school stakeholders. 


Target Audience | Administrators 


Needs Frequent and timely communication with local school stakeholders 


Initiatives Provide specific and timely communication related to fluidity with COVID and 
online learning 
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Goals 


Actions 


CSFs 


KPIs 


To communicate decision-making rationale and processes with local school 
stakeholders 


e Identify existing communication strategies and channels 

e Explore possibilities for ongoing communication with school-level 
stakeholders 

e Solicit feedback from school-level stakeholders 

e Evaluate effectiveness of communications 


Schools receive timely communication as COVID fluidity may influence decision- 
making 


e Frequency of regular communication channels and flows 
e Percentage of resources aligned to communication channels 
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Subdimension: Strategies 


Survey results indicate broad satisfaction with the institution’s strategic planning and alignment for online 
learning across administrators, staff, and teachers, both pre- and post-COVID. One prominent finding across all of 
the survey items related to this subdimension relates to the increased satisfaction post-COVID. What’s more, of the 
five questions posed for this subdimension, four revealed satisfaction rates above 70% post-COVID across all three 
role groups. 


With respect to the extent to which “a mission, vision, and strategic plan is developed addressing the importance 
of quality in online learning,” respondents expressed 63.5% satisfaction pre-COVID, which increased to 70.1% 
post-COVID. Satisfaction rates were relatively stable among role groups (Figure 4). 


A MISSION, VISION, AND STRATEGIC PLAN IS DEVELOPED 
ADDRESSING THE IMPORTANCE OF QUALITY IN ONLINE 
LEARNING. 


mAdministrators mStaff mTeachers 
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a 

STRONGLY DISAGREE 76356194 

Figure 4 


Survey respondents also indicated satisfaction with the ways in which the institution’s strategy for online learning 
is articulated and updated regularly, and the extent to which it incorporates emerging educational and business 
trends and requirements. A majority expressed satisfaction both pre-COVID (66.1%) and post-COVID (71.4%) 
(Figure 5). As with the other facets of this subdimension, satisfaction increased post-COVID. 
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AFTER-COVID 


PRE-COVID 


A STRATEGY FOR ONLINE LEARNING IS ARTICULATED AND 
UPDATED REGULARLY INCORPORATING EMERGING 
EDUCATIONAL AND BUSINESS TRENDS AND REQUIREMENTS. 
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Strategic alignment between an organization’s strategy and its goals is imperative for large scale leadership in 
education. A majority of survey respondents indicated satisfaction both pre- and post-COVID in this area, with 
70.7% of respondents agreeing that leadership has developed goals that align with the strategy for online 


learning, post-COVID (Figure 6). 
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Figure 6 


STRATEGY FOR ONLINE LEARNING. 
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Likewise, a majority of respondents agreed that leadership identifies measures of goals and collects timely data to 
ensure quality and identify areas of improvement (66.4% pre-COVID; 70.5% post-COVID) (Figure 7). 


LEADERSHIP IDENTIFIES MEASURES OF GOALS AND COLLECTS 
TIMELY DATA TO ENSURE QUALITY AND IDENTIFY AREAS OF 
IMPROVEMENT. 
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Figure 7 


While all of the items indicated a majority sense of agreement with respect to leadership’s strategic alignment, 
one item generated noticeably smaller majority. That is, when surveyed on the extent to which the institution’s 
strategic plan for online learning aligns with other strategic plans (e.g., academic, technology) within the school or 
district, the pre-COVID agreement across all groups was just 57.4%. However, this agreement did increase to 
64.3% post-COVID (Figure 8). 


THE STRATEGIC PLAN FOR ONLINE LEARNING ALIGNS WITH 
OTHER STRATEGIC PLANS WITHIN THE SCHOOL OR DISTRICT. 
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Interestingly, pre-COVID both administrators and teachers responded with disagreement above 20%, but both 
groups saw a post-COVID decrease to 16.3% for administrators and 16.7% for teachers. 


The triangulated interview data did seem to support the lower rate of agreement related to the survey item about 
strategic alignment between plans for online learning and other strategic plans within schools and districts. For 
example, one respondent indicated that while a plan for online learning did exist at their school prior to COVID, 
the resources and time needed to allocate toward building out the details of that plan were placed elsewhere. 
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Recommendation: Create alignment with and between the institution’s strategic plan for 
online learning with other strategic planning components. 


Target Administrators 
Audience 
Needs Articulation of and resource allocation for strategic alignment between online learning plans 


and other strategic planning components 
Initiatives Comprehensive strategic planning to account for any discrepancies or disparities between 
planning goals and organizational alignment 
Goals To articulate a coherent, congruent strategic plan that accounts for online learning in a way 
that aligns with other priorities and plans 
Actions e Review current strategic planning across the organization -- including but not limited 
to online learning, technology, and assessment. 
e (Re)Establish strategic planning team and schedule regular reviews for congruences 
and discrepancies across strategic planning efforts 
e Establish congruent and aligned strategic plan, accounting for online learning 
CSFs Sensemaking and synthesizing planning at various school sites and contexts 
KPIs e Strategic plan action items are aligned with short- and long-term goals 
e Strategic plan is communicated to relevant stakeholders 


Subdimension: Policies 


At a policy level, survey items for this subdimension reveal a strong sense of agreement from respondents that 
leadership has enacted policies that have been reviewed and updated to encourage online learning, taking care to 
address issues of academic integrity and to support online teaching, learning and innovation. Broadly, 
respondents along each item in the subdimension increased their sense of agreement from pre-COVID to post- 
COVID, and responses across role groups remained fairly consistent. 


The majority of survey respondents across all three role groups agreed that policies and procedures for online 
learning are reviewed and updated regularly. Further, this question saw a notable jump from the pre-COVID 
agreement (61.4%) to post-COVID (70.4%), suggesting that stakeholders noticed leadership’s responsiveness to 
the pandemic and policies related to online learning (Figure 9). 
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REVIEWED AND UPDATED REGULARLY. 
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Survey results also indicated that a majority of respondents agreed with the notion that leadership established 


policies that encourage the development of online classes and programs (e.g., a policy for intellectual property of 
class materials). This was true both pre-COVID (62.9%) and post-COVID (68.2%), with all three role groups 
responding with increased agreement from pre- to post-COVID (Figure 10). 
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AND PROGRAMS. 


mAdministrators mStaff m Teachers 


14.1% 13.3% 13.4% 


55.2% 


56.6% 54.4% 


19.3% 18.4% 20.7% 
7.7% 6% 5% 
LAY) 4 


0.8%11.0%10.9% 


51.9% 
19.5% 
11.0%9.9%10.6% J 


55.1% 51.7% 


21.5% 


22.4% J 


a 


23 


FOC] oun pears 


Like previous items for this subdimension, a majority of survey respondents agreed that leadership pursued 
policies which address academic integrity of online class assignments and assessments. While the pre-COVID 
majority was among the smallest of the items for this subdimension (59.9%), it made a significant jump to 67.3% 
post-COVID (Figure 11). 


POLICIES ADDRESS ACADEMIC INTEGRITY OF ONLINE CLASS 
ASSIGNMENTS AND ASSESSMENTS. 
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With respect to the promotion policies toward teachers and staff including criteria supporting online teaching, 
learning, and innovation, there was majority agreement from all three respondent groups both pre- and post- 
COVID. It is notable that this item had the lowest agreement pre-COVID at 54.1%, with 52.3% of teachers agreeing 
or strongly agreeing with the item, while administrators (58.0%) and staff (60.4%) agreed at a higher rate. 
However, post-COVID, all three groups responded with a higher rate of agreement at 59.6%. Still, teachers had the 
lowest rate of post-COVID agreement (57.8%), while both administrators and staff had agreement responses 
above 60%) (Figure 12). 
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PROMOTION POLICIES OF TEACHERS AND STAFF INCLUDE 
CRITERIA SUPPORTING ONLINE TEACHING, LEARNING, AND 
INNOVATION. 
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Narrative data from the teacher interviews tended to indicate that teachers vary in their views on the extent to 
which policies toward them offered support for online teaching, learning, and innovation. For example, one 
teacher noted that their school needs better training for the teachers and technical support for the teachers and 
students, while another suggested that these things are in place and strongly supported. 


Recommendation: Explore policies that aim to improve quality in online teaching, 
learning, and innovation. 


Target Administrators 

Audience 

Needs Policy environment that improves quality in online teaching, learning, and innovation 

Initiatives Policies that meet the needs of quality and innovative online teaching and learning 

Goals To implement appropriate processes to support online teaching and learning 

Actions e Review existing policies related to online teaching, learning, academic integrity, and 
innovation 


e Convene stakeholders to explore potential improvement to policies 

e Articulate updated policies toward quality online teaching and learning 

e Ensure full compliance with the nationally/locally approved curricula and 
teaching/learning outcomes 


CSFs Effective implementation of online teaching and learning 
KPIs Creation and dissemination of updated policies for quality in online teaching, learning, and 
innovation 
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Subdimension: Process 


The subdimension of leadership processes largely followed the trends of the other subdimensions, with each 
survey item yielding a majority of agreement across administrators, staff, and teachers -- with agreement 
percentages for each item increasing from pre- to post-COVID. However, the agreement percentages for 
leadership processes related to teacher evaluations and for resource allocation pre-COVID had the slimmest 
majority agreement, both at approximately 54%. While both jumped to over 60% across all three role groups for 
post-COVID responses, the interview data show that both administrators and teachers have identified some gaps 
in leadership processes to evaluate teachers and account for resource allocation. After identifying some item 
responses below, recommendations for this leadership processes subdimension will focus on processes for 
teacher evaluation, as resource allocation is its own leadership subdimension and will be discussed in the 
following section. 


With respect to the survey data on processes which exist to provide professional development, training, support, 
and evaluation of online learning, the majority of respondents agreed for both pre-COVID (62.9%) and post-COVID 
(69.5%), with agreement increases of several percentage points across all three role groups (Figure 13). 


PROCESSES EXIST TO PROVIDE PROFESSIONAL DEVELOPMENT, 
TRAINING, SUPPORT, AND EVALUATION OF ONLINE LEARNING. 
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Related to a processes subdimension survey item inquiring about the extent to which evaluations are conducted 
periodically by qualified and objective external parties who have a demonstrated ability to deliver a 
comprehensive review of internal processes, all three role groups answered with majority agreement both pre- 
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COVID (54.6%) and post-COVID (61.3%). As with the previous item, respondents returned a noticeable increase for 


their post-COVID assessment of leadership processes (Figure 14). 


EVALUATIONS ARE CONDUCTED PERIODICALLY BY QUALIFIED 
AND OBJECTIVE EXTERNAL PARTIES WHO HAVE A 
DEMONSTRATED ABILITY TO DELIVER A COMPREHENSIVE 
REVIEW OF INTERNAL PROCESSES. 
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The final processes subdimension survey item asked respondents to gauge the extent to which processes for 
planning and allocating resources for online learning, including financial resources, in accordance with the 


strategic plan have been established. In alignment with the other responses for leadership processes, the cohorts 


collectively affirmed these processes were in place, by a small majority pre-COVID (54.4%), and by an increased 
majority post-COVID (61.1%) (Figure 15). 
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A PROCESS FOR PLANNING AND ALLOCATING RESOURCES FOR 
ONLINE LEARNING, INCLUDING FINANCIAL RESOURCES, IN 
ACCORDANCE WITH THE STRATEGIC PLAN IS ESTABLISHED. 
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Recommendation: What the foregoing data suggests is that a small majority of 
respondents across the groups agreed that the leadership processes have been in place to 
support teacher evaluation and resource allocation. While all three role groups answered 
in majority agreement, staff tended to have a more optimistic view than administrators 
and teachers, often by five to seven percentage points. This suggests that both 
administrators and teachers could stand to receive more clarity in the processes for 
evaluation, and the interview data seems to corroborate that sentiment. This largely 
makes sense at a human level, as administrators are often charged with conducting 
performance evaluations, and teachers are the subjects of those evaluations. 


Target Administrators, Teachers 

Audience 

Needs Clarity on processes for evaluating teacher performance in online teaching 

Initiatives Processes that clarify teacher performance evaluation 

Goals To create clear processes for evaluating teacher performance 

Actions e Review existing processes and modify for evaluating quality of online teaching 
performance 


e Identify and develop additional processes needed to support teachers 
e Consider feedback on the existing policies and introduced initiatives from all 
stakeholders to consult interests of the entire education community 
CSFs Teachers delivering quality online instruction, with both administrators and teachers 
understanding the criteria and quality standards for teacher performance 


28 EE EE as 


KPIs e = Short term: Clearly articulated teacher evaluation processes 
e Longterm: Teacher evaluation results 


Subdimension: Resource Allocation 


On the leadership subdimension of resource allocation, respondents across all three role groups again responded 
to survey items with a majority of agreement on institution leadership’s support and efforts, though with smaller 
pre-COVID majority percentages of agreement or strong agreement than previous subdimensions. Interview data 
with teachers and administrators often pointed to resource allocation disparities among urban and rural schools, 
as well as those that were identified to receive technology, devices, as well as the extent to which all students have 
Internet access at home. All told, the data suggest that the institution’s support and efforts toward schools during 
COVID yielded positive outcomes along the resource allocation subdimension. 


On the subdimension question of sufficient resource allocation, including financial resources, to effectively 
support the mission of online education is demonstrated, a small pre-COVID majority (54.1%) grew to 60.0% in 
post-COVID responses (Figure 16). 


SUFFICIENT RESOURCE ALLOCATION, INCLUDING FINANCIAL 
RESOURCES, TO EFFECTIVELY SUPPORT THE MISSION OF 
ONLINE EDUCATION IS DEMONSTRATED. 
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When questioned about the extent to which “incentives and/or financial compensation are provided to teachers 
and staff for professional development and instructional design activities,” respondents offered a slim majority of 
agreement, including both responses of “agree” and “strongly agree,” pre-COVID (50.1%). Notably, among 
teachers and administrators, the pre-COVID percentage indicating disagreement was above 30% for both groups, 
the highest of any survey item in the leadership dimension. However, disagreement dropped and overall 
agreement across groups increased to 55.7% total post-COVID (Figure 17). This item will be discussed further in 
the recommendations. 
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INCENTIVES AND/OR FINANCIAL COMPENSATION ARE 
PROVIDED TO TEACHERS AND STAFF FOR PROFESSIONAL 
DEVELOPMENT AND INSTRUCTIONAL DESIGN ACTIVITIES. 
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The majority of survey respondents across all three role groups agreed that resources are adequately allocated 
and projected to sustain and scale online learning, both pre-COVID (54.0%) to 61.3% post-COVID. Notably, among 
administrators, a pre-COVID disagreement response rate of 27.1% fell to 19.1% post-COVID, indicating that 
institution efforts during the pandemic have been effective in providing resources to sustain and scale online 
learning. See Figure 18 for item responses across all three role groups. 
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SUSTAIN AND SCALE ONLINE LEARNING. 
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The final survey item for this subdimension sought to understand stakeholders’ perceptions about investment in 
instructional personnel and staff time to support online learning and instructional technologies. While pre-COVID 
responses indicated a majority agreement of 58.0%, post-COVID response rates increased to 63.8% (Figure 19). 
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INSTRUCTIONAL TECHNOLOGIES. 
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Recommendation: Taken wholly, respondents largely agreed in majority that leadership 
decisions and practices related to resource allocation have served to support, sustain, 
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and scale the mission of online education. In that light, two opportunities emerge from 
the survey and interview data to further strengthen leadership resource allocation. First, 
leaders can explore the extent to which resources are distributed equitably across 
schools, both urban and rural. Second, leadership may seek opportunities to address 
resource allocation related to incentivizing or providing financial compensation for 
teacher training and instructional design professional learning. 


Target Administrators 
Audience 
Needs e Equitable resource allocation (physical devices, teacher training) across urban and 


Initiatives e 
e 
Goals e 
e 
Actions e 
e 
CSFs e 
e 
KPIs e 


rural schools 
Incentives and/or compensation for teacher training and professional learning 
related to online instruction 
Explore opportunities for equitable resource allocation for schools 
Processes that incentivize teacher participation in training and professional learning 
for online teaching 
To create more equitable resource allocation for schools across geographies 
To further incentivize teachers to participate in professional learning for online 
teaching 
For equitable resource allocation: 

© Explore current resource allocation policies and processes to identify 

potential gaps between school designations 

o Establish plan to remediate any identified gaps 
For incentivizing teachers to participate in professional learning: 

o Establish core and/or essential offerings for teacher professional learning 

related to online teaching 

o Identify and share funding opportunities or incentives with teachers 
Capacity to deliver more equitable distribution of physical devices and resources 
Teachers’ responses to incentives for professional learning 
Equitable access to resources across school contexts 
Teacher participation in professional learning 


32 Ss 


Subdimension: Staffing & Innovation 


On the leadership subdimension of staffing and innovation, survey respondents largely affirm institution 
leadership’s efforts to provide innovative and supportive staff to meet strategic online learning goals. Across all 
subdimension survey items, all three role groups shared majority agreement in both their pre- and post-COVID 
responses. Not surprisingly, staff responses to each item ranged approximately 10-15% higher than administrator 
and teacher responses across survey items, but that can be interpreted as a kind of self-referendum on their 
performance both pre- and post-COVID. Narrative data from interview subjects shows broad corroboration for the 
survey data, with several interviewees praising the work of the support staff, or simply expressing a wish for even 
more support due to the quality of support staff interactions. 


Largely, respondents agreed that sufficient qualified professional, administrative, and support staff are provided 
to meet online learning strategic goals, with 60.7% agreeing or strongly agreeing pre-COVID. That percentage 
jumped to 68.5% post-COVID. While staff respondents answered in a higher percentage, all three role groups 
affirmed their agreement with this item (Figure 20). 


SUFFICIENT QUALIFIED PROFESSIONAL, ADMINISTRATIVE, 
AND SUPPORT STAFF ARE PROVIDED TO MEET ONLINE 
LEARNING STRATEGIC GOALS. 
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Survey respondents also largely agreed that technical staff are provided to support the learning environment and 
the technology needs of students and teachers. An initial, pre-COVID agreement of 63.3% across all three role 
groups increased to 69.8% post-COVID, with both administrators and staff responding with agreement rates above 
70% (Figure 21). 
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TECHNICAL STAFF ARE PROVIDED TO SUPPORT THE LEARNING 
ENVIRONMENT AND THE TECHNOLOGY NEEDS OF STUDENTS 
AND TEACHERS. 
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The final item on the staffing/innovation subdimension sought to understand the extent to which respondents 
agreed that the institution incorporates an expectation of innovation in teaching and learning through staff 
evaluations and promotions. Pre-COVID, the agreement rate was 60.2% across all role groups, and that increased 
to 66.1% post-COVID (Figure 22 
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LEADERSHIP INCORPORATES AN EXPECTATION OF 
INNOVATION IN TEACHING AND LEARNING THROUGH STAFF 
EVALUATIONS AND PROMOTIONS. 
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Figure 22 


Expert Recommendation: Overall, both survey respondents and interviewees for the 
staffing/innovation dimension reported satisfaction with the staffing and innovation 
across their schools. Our expert recommendations are for leadership to continue to 
pursue opportunities and incentives for innovation and outstanding service provision on 
the part of staff members. 


Target Administrators 

Audience 

Needs Expanded drive and incentivization for innovation on the part of administrators, staff, and 
teachers 

Initiatives Create pathways for innovation in online teaching and learning 

Goals To enhance a culture of innovation and service across school level stakeholders 

Actions e Establish regular opportunities and light incentives for innovation 


e Identify barriers to innovation and “pain points” and/or “fear factors” that may 
prevent innovative behavior on the part of administrators, staff, and teachers 
CSFs School level stakeholders feel safe seeking to innovate without reprisal or punishment 
KPIs Number of school-level stakeholders who show increased innovative behaviors 


Dimension II: Curriculum Design & 
Planning 


The Curriculum Design and Planning dimension measures five subdimensions including Instructional Design 
Methods & UDL, Learning Objectives, Alignment, Course Materials and Content, and Innovation. 


A high degree of readiness regarding curriculum design and planning was observed in overall survey results, 
potentially due to existing efforts to increase online education in K-12 environments as well as the curriculum 
responsibilities of the Ministry. Study results do suggest that despite this readiness prior to the pandemic, the shift 
to emergency remote teaching revealed pedagogical considerations across this dimension that were not evident, 
or perhaps not necessary for curriculum prior to COVID. Attending to the recommendations provided is an 
opportunity to support readiness and continuous improvement moving forward. 


Subdimension: Instructional Design Methods & 
Universal Design for Learning (UDL) 


Survey data indicates that data suggests that across all three role groups, there was agreement that the 
implementation of curriculum developed by the Ministry of Education is a core responsibility for teachers. 
However, across all three groups, the agreement rate for this subdimension item fell from pre-COVID (80.6%) to 
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post-COVID (75.2%). What this suggests is that the pandemic likely revealed important pedagogical considerations 
that may not have been originally included in the curriculum, including how to maintain relationships with 
students during emergency remote teaching (ERT), as well as the social-emotional needs that learners may have. 
See Figure 23 for more detail related to the item response. 


IMPLEMENTATION OF CURRICULUM DEVELOPED BY THE 
MINISTRY OF EDUCATION IS A CORE RESPONSIBILITY FOR 
TEACHERS. 
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With respect to instructional design, respondents across all three role groups of administrators, staff, and teachers 
agreed that the instructional design of online classes is supported by teachers, instructional designers, technical 
experts, and evaluators. Through interview data, several teachers cited the Ministry of Education as a central 
source for instructional design, though local schools have educational resource and support staff who can provide 
technical assistance. 


However, two questions posed only to teachers revealed an opportunity for creating clarity for teachers around 
the instructional design choices for the curriculum. When asked about their agreement with the extent to which 
instructional design focuses on authentic and active learning that is student-centered creating opportunities for 
students to interact with each other and the teacher, respondents’ agreement decreased from 74.2% pre-COVID to 
68.8% post-COVID. While the agreement rate is still a predominant majority of teachers, the corroborating 
narrative data does suggest that teachers could stand to have more clarity around the instructional design 
frameworks and philosophy for how curricula are created. See Figure 24 for more details. 
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THE INSTRUCTIONAL DESIGN FOCUSES ON AUTHENTIC AND 

ACTIVE LEARNING THAT IS STUDENT-CENTERED CREATING 

OPPORTUNITIES FOR STUDENTS TO INTERACT WITH EACH 
OTHER AND THE TEACHER. 
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With respect to Universal Design for Learning (UDL), a framework to improve and optimize teaching and learning 


for all individuals, a survey item asked respondents about the extent to which instructional materials are quality 
materials and accessible to all students, easy to use, and accessible on any Internet-ready device. While 
administrators were slightly less likely than their staff and teaching colleagues to agree, particularly pre-COVID, 
the overall group responses showed a majority of agreement, both pre-COVID (60.8%) and post-COVID (65.6%). 
See Figure 25 for more details. Similarly, respondents also agreed that online classes are designed based on 
research-design principles and accessibility guidelines with consideration for usability, including ease of 
navigation. Narrative data suggests that teachers frequently provide feedback back to the Ministry of Education 
on curricula, and that teachers sense that feedback is reviewed with care and the curricula is updated. 
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INSTRUCTIONAL MATERIALS ARE QUALITY MATERIALS AND 
ACCESSIBLE TO ALL STUDENTS, EASY TO USE, AND ACCESSIBLE 
ON ANY INTERNET-READY DEVICE. 
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Figure 25 


Recommendation: Based on the survey data and interview responses from teachers, there 
is a significant opportunity to utilize and articulate the instructional design decisions, 
frameworks, and philosophies that guide the K-12 curriculum design across the Kingdom. 
Teachers did share that they believe they receive high quality materials, so adding some 
pedagogical and design context around those materials may serve to strengthen their 
delivery of quality online learning experiences for students. 


Target Ministry of Education 

Audience : 7 ; a 7 

Needs Utilize and articulate to school stakeholders - teachers, in particular - the specific instructional 
design frameworks, including UDL, in the design of curricula 

Initiatives Embrace instructional design practices like UDL explicitly in the design of curricula 

Goals To disseminate awareness and importance of instructional design considerations of UDL 

Actions e Review the existing frameworks used for instructional design 


e Identify with instructional designers and curriculum developers the most appropriate 
frameworks going forward with strong support for UDL 
e Introduce stimuli for increasing teacher motivation for innovative tech-based practice 
e Develop means for proactive pedagogue retention 
CSFs Quality online teaching and learning across K-12 schools in the Kingdom 
KPIs Percentage of teachers who report better preparation to deliver curricula, regardless of the 
fluidity of face-to-face, hybrid, or online instruction 


38 SS S= oS 


Subdimension: Learning Objectives 


Along the subdimension of learning objectives, survey data revealed that administrators, staff, and teachers 
largely agree (65.3% pre-COVID; 69.2% post-COVID) that measurable learning objectives are provided for each 
online class and clearly state what the learner will be able to demonstrate as a result of successfully completing 
the class. See Figure 26 for details. Further, all three groups agreed that, both pre- and post-COVID, that a 
development process is followed that ensures online classes are designed so that students develop the necessary 
knowledge and skills to meet measurable learning objectives. Interview data largely corroborates these findings, 
with one teacher noting that “the teacher is able to design their teaching or their journey based on the learning 
objectives” for each course. 


MEASURABLE LEARNING OBJECTIVES ARE PROVIDED FOR 
EACH ONLINE CLASS AND CLEARLY STATE WHAT THE LEARNER 
WILL BE ABLE TO DEMONSTRATE AS A RESULT OF 
SUCCESSFULLY COMPLETING THE CLASS. 
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Expert Recommendation: While the survey and interview data suggest that measurable 
learning objectives and course designs are firmly in place to ensure that students develop 
the necessary knowledge and skills to meet measurable learning objectives, the 
opportunity exists for teachers to engage students in building meaningful understanding 
and intrinsic motivation to achieve these learning outcomes. 


Target Teachers 
Audience | 
Needs _ Help students understand and make meaningful connections to the learning outcomes and 


__ their required performance to meet those objectives 


Initiatives Create opportunities for student participation in meaning-making around course learning 
objectives, and to articulate what success means in a way that they can understand and begin 
to own 

Goals To guide students to articulate for themselves what the learning objectives mean and to know 

eos | how they can assess their own progress toward those learning objectives 

Actions e Teachers take time at the beginning of courses, terms, and content units to review 

course learning objectives 
e Teachers ask students to create shared understandings of course learning objectives 
and what success means in relation to those 

CSFs _ Students and teachers are engaged in high quality online teaching and learning 

KPIs _ Student mastery of course learning objectives 


Subdimension: Alignment 


With respect to the alignment subdimension of Curriculum Design & Planning, administrators, staff, and teachers 
largely agreed that guidelines regarding standards for class development, and instructional design and delivery of 
online classes are used to create alignment. One data point worth noting, however, is that teachers saw a slight 
decrease in the extent to which they agreed that content and learning activities aligned with the assessments and 
learning objectives, moving from 69.6% agreement pre-COVID to 67.5% post-COVID. Narrative data from 
interviews suggest that the uniqueness of the challenges represented by the emergence of COVID-19 may largely 
account for that dip, but overall a majority of teachers agree on the alignment of standards to curriculum and 
instruction. 
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Expert Recommendation: While there does seem to be a solid sense of alignment around 
learning objectives and curriculum and instruction with stakeholders, there is further 
opportunity for teachers to incorporate supports to ensure that standards-based 
curriculum and instruction is aligned to learning outcomes, as well as being accessible 
and understandable to each student. 


Target Teachers 
Audience ! 
Needs Alignment between content and learning activities to the assessments and learning objectives 
in online instruction 
Initiatives Teachers clarify learning outcomes that align with guidelines regarding standards for 
instructional planning 
Goals To create alignment between learning activities and the assessments and learning objectives 
A in online instruction 
Actions School teacher teams hold regular meetings to peer review alignment of learning outcomes to 
curriculum and instructional planning 
CSFs Regular communications from teachers back to the Ministry of Education/NELC sharing their 
learnings 
KPIs Percentage of learners that report that instruction, assignments, and content meets their 
needs 


Subdimension: Course Materials & Content 


For the subdimension of course materials and content, teachers were surveyed on the extent to which online 
classes included content -- both instructional and operational/functional for the course -- that was aligned to the 
assessment and learning objectives. Taken wholly across content types, teachers displayed a predominant sense 
of agreement. With respect to syllabi, for example, teachers did agree that online classes include a syllabus ora 
document on the class site outlining learning objectives, assessment methods, and descriptions of learning 
activities, including content (64.3% pre-COVID; 65.9% post-COVID). 


Teachers also agreed that content, including reading assignments and lectures, aligns with the assessments and 
learning objectives, and these materials offered content that is current, rich, and sufficient in breadth and depth. 
Moreover, teachers also agreed that the content used includes materials that are accessible and interactive, such 
as Open Education Resource (OER) (64.8% pre-COVID; 67.8% post-COVID). In interviews, teachers shared a wide 
variety of content sources available to them through Ministry supports, though some also acknowledged the 
burden of time and skill required to continually create engaging content for online learning on their own. 
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Recommendation: Teachers have articulated through survey and interview that they feel 
broadly supported with meaningful content and material support. As such, it is 
recommended that the Ministry of Education regularly review course materials and 
content for sufficient coverage and currency, and update as appropriate with current and 
media-rich content. 


Target Ministry of Education 
Audience 
Needs Ensure that course materials and content included are aligned to assessment and learning 


objectives, are of a high quality, accessible, and are regularly updated with current and media- 
rich content 


Initiatives e Regularly review course materials and content 
e Regularly update course materials and content with current and media-rich sources 
Goals To continually ensure that course materials are aligned to assessment and learning objectives, 
are of a high quality, accessible, and are regularly updated 
Actions e Develop regular review and updating processes and timelines for course content and 
materials 


e Seek opportunities to promote OER in course materials 
CSFs Course content and materials are regularly reviewed for currency and resource content and 
updated as appropriate 
KPIs e Percentage of course content and materials reviews completed 
e Number of resources updated with current and media-rich content 


Subdimension: Innovation 


With respect to the innovation subdimension of Curriculum Design & Planning, the survey asked respondents to 
gauge the extent to which they agreed with the statement that teachers and staff are supported and encouraged 
to incorporate pedagogical and technological methods to improve student outcomes through curricular redesign. 
Overall, a solid majority of the total respondents across the role groups agreed with the statement, both pre- 
COVID (68.1%) and post-COVID (70.1%). 


One notable aspect of the survey responses was the distinction between administrator and staff agreement, and 
with teacher agreement. For example, post-COVID, administrators (76.1%) and staff (75.0%) agreed or strongly 
agreed that teachers and staff are encouraged to innovate, while teachers agreed at a rate of 68.0%. Narrative 
data did not reveal consistent themes of agreement, though one teacher respondent did comment that 
innovation tended to be localized at the school level (Figure 27). 
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TEACHERS AND STAFF ARE SUPPORTED AND ENCOURAGED TO 
INCORPORATE PEDAGOGICAL AND TECHNOLOGICAL METHODS 
TO IMPROVE STUDENT OUTCOMES THROUGH CURRICULAR 
REDESIGN. 
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Expert Recommendation: While the data does not reveal dissatisfaction on the part of 
teachers with respect to their sense of encouragement to innovate, the discrepancy 
between their perceptions and those of administrators and staff suggests an opportunity 
for teachers to expand their efforts to innovate and redesign their curriculum. As one 
teacher noted, “I think that students have greater aspirations than the (current) 
curricula,” suggesting that both students and teachers might be poised to further 
innovate. Thus, it is recommended that school leaders continue to expand opportunities 
to collaborate and a cultural permission to redesign their curricula. 


Target Administrators 

Audience 

Needs Expand the opportunity set for teachers to innovate with their curricular and/or instructional 
design 


Initiatives | Encourage teachers to be more pedagogically or technologically innovative 


Goals To promote a supportive climate and culture for teacher innovation with respect to curricular 
designs and redesigns, such as using design thinking 


Actions e Incorporate innovation efforts as a metric for performance evaluations 

e Explore possibilities for incentivizing or promoting innovative design efforts (e.g. regular 
sharing opportunities at professional learning or team meetings; celebrating “what 
didn’t work” as a way to workshop innovative lesson improvements; solicit submissions 
for a periodic/regular innovation award) 


CSFs Teachers feel supported and encouraged to innovate in their instructional design 
KPIs Reported enthusiasm to new lessons by students and teachers 
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Dimension III: Online Teaching & 
Learning 


The Online Teaching and Learning dimension measures seven subdimensions including Communication, 
Engagement, Expectation Setting, Outcomes, Course Interaction, Feedback, and Innovation. 


Readiness in online teaching and learning, like previous dimensions, reflected preparedness pre-COVID, with both 
increases in some areas post-COVID that indicate prompt attention to needs as well as some areas of existing or 
new needs illuminated by the pandemic. Of particular note regarding a need for increased readiness moving 
forward in this dimension are course interaction and feedback. 


Findings related to online teaching and learning indicate that there were slight increases in the communication, 
engagement, and course interaction subdimensions, and slight decreases in outcomes and feedback post-COVID. 
These elements are critical to online teaching and learning, and these slight increases suggest that stakeholders 
may have made changes to support students online following the pandemic. Recommendations include a number 
of items that support continuous improvement in online teaching and learning, as well as provide a solid 
foundation for increasing effectiveness and capacity. 


Subdimension: Communication 


In online teaching and learning, students rely on accurate and timely communication with their teachers. If this 
does not happen, it can be very easy for the students to feel isolated and struggle in the class. Providing 
opportunities for communication can help to mitigate this issue. Administrators, staff and teachers were asked to 
rate expectations about communication (virtual synchronous sessions, email, class news and announcements, 
online discussions, and assignment feedback). The majority (63.1% Administrators; 68.0% Staff; 60.6% Teachers) 
were in agreement that this did occur prior to COVID-19. After COVID-19, those in agreement increased (75.0% 
Administrators; 74.5% Staff; 71.4% Teachers) reflecting changes that may have been made to support students 
during the transition to remote learning (Figure 28). 
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TEACHERS ARE EXPECTED TO HOLD VIRTUAL SYNCHRONOUS 
CHECK-INS WITH STUDENTS AND FREQUENTLY COMMUNICATE 
WITH THEIR STUDENTS VIA EMAIL, CLASS NEWS AND 
ANNOUNCEMENTS, ONLINE DISCUSSIONS, AND ASSIGNMENT 
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FEEDBACK. 
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To ensure that appropriate levels of communication are occurring, many institutions will set standards for 
teachers, including response times and contact information that needs to be provided. Based on survey results 
reflecting agreement with this (62.4% Administrators; 67.7% Staff; 60.2% Teachers), it appears that many schools 
had set this expectation before COVID-19. During the transition to remote learning, it seems likely that more 
adopted this expectation since the numbers of those in agreement increased (72.9% Administrators; 74.3% Staff; 


68.8% Teachers). 


Expert Recommendation: Develop processes and policies for expectations around 


communications. 
Target Teachers 
Audience 
Needs Policies and processes for expectations around communications 


Initiatives Establish expectations for responsiveness 

Goals To set expectations regarding when students can expect to communicate with the teacher 

Actions Develop, define and communicate expectations for when students can expect responses (to 
emails, phone calls, etc.) from the teacher 

CSFs Established parameters for when teachers will respond to student communications 

KPIs Teacher response time 
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Subdimension: Engagement 


Student engagement in the online class is one method that can help support success. This can be accomplished 
by creating a classroom environment that is both academically challenging and provides social interactions. Prior 
to COVID-19, administrators (62.5%), staff (67.6%), and teachers (61.0%) selected Agree or Strongly Agree on the 
survey to reflect that their school was already doing this. Responses reflected a slight increase in agreement 
(70.0% Administrators; 71.1% Staff; 66.0% Teachers) after the transition to remote learning (Figure 29). 


ONLINE CLASS DESIGN PROMOTES STUDENT ENGAGEMENT, 
INCLUDING ACADEMIC CHALLENGE AND SOCIAL 
INVOLVEMENT WITH EACH OTHER AND THE INSTRUCTOR. 
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The interviews that were conducted with teachers reflected some concerns, especially during COVID-19 with being 
able to communicate with their students. “I asked the principal if | could send my students some assignments via 
My Opera, she said we’re not allowed.” Another teacher indicated that, “To be honest with you, | contacted my 
students via Whatsapp and wanted to contact them also via Instagram, but the head mistress told me not to and 
that it will cause me problems. All rules say not to contact students. My head mistress even asked me to stop 
communications with them, and | honestly didn’t.” 


However, not all teachers had this issue with communicating with their students. In many cases, teachers made an 
effort to increase communication or others methods to engage their remote learning students. “Whenever a 
student asks a question, | answer her immediately. It all depends on the type of activities | present. It shows me 
whether they’re attentive or not.” Another teacher found that online tools helped to increase the communication 
and engagement with students that had been more passive in the face-to-face classroom, “But after the first week 
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has passed, and | saw students’ eagerness and communication via e-mail, etc. Students were more carefree. | 
noticed that some students who were very shy and don’t usually participate tend to do so now. Maybe because 
they have 2 options, either by speaking or writing. Some students, | never expected from them to participate just 
how they do at class, but they surprised me by their creativity. Maybe one of the reasons is that the student feels 
comfortable at home, hence he was positively affected.” 


Recommendation: Provide engagement opportunities, both formal and informal, for 
students to interact with one another and teachers, and ensure that academic challenge 
is incorporated into these experiences as appropriate. 


Target Teachers, Students 

Audience 

Needs Increase opportunities for students to interact with their fellow students and the teacher 
Initiatives Implement both formal and informal opportunities for students to interact with each other 


and the teacher 
Goals To increase student engagement with course content, their peers, and the teacher through 
academically challenging components 


Actions Include intentional opportunities for academically challenging interactions with students and 
the teacher in course design (formal) and delivery (informal) 

CSFs Stronger connections between the students and the teacher 

KPIs e «Quality course reviews 


e Presence of academically challenging interactive opportunities 

e Percentage of students who indicate that interactive opportunities are present and 
academically challenging 

e Teacher satisfaction 


Subdimension: Expectation Setting 


Another method teachers can use to increase the potential for student success is by providing clear expectations 
for learners. This can start by providing clear, concise, and useful assignment instructions. The results reflected 
agreement that this was occurring at many schools prior to COVID-19 (65.6% Administrators; 69.3% Staff; 64.2% 
Teachers). The results showed a slight improvement for after COVID-19 (72.6% Administrators; 73.2% Staff; 69.2% 
Teachers). During the transition to remote learning, it is likely that a change in processes occurred to provide 
clearer instructions in order to better support student learning (Figure 30). 
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ONLINE CLASSES CONTAIN CLEAR, CONCISE, AND USEFUL 
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ASSIGNMENT INSTRUCTIONS. 
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Providing expectations for when students should be participating in the class and when the teacher will provide 


feedback helps to establish standards that encourage and support success in the online classroom. As shown in 


Figure 31, the majority of teachers indicated that they were providing these expectations in a syllabus prior to 
COVID-19 with only a slight increase in teachers indicating this after COVID-19. In addition, teachers indicated that 


they were eliminating barriers to success by providing clear expectations for participation and performance. Prior 


to COVID-19, 62.4% of teachers agreed or strongly agreed with this statement and after COVID-19, there was a 


slight increase in agreement to 68.0% (Figure 32). 
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AFTER-COVID 
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EXPECTATIONS FOR STUDENT ASSIGNMENT COMPLETION, 
INCLUDING CLASS SCHEDULE WITH DEADLINES, GRADING 
SCHEME AND ASSOCIATED POLICIES, AND TEACHERS' 
FEEDBACK ARE PROVIDED IN THE CLASS SYLLABUS. 


STRONGLY AGREE 
AGREE 

NEUTRAL 

DISAGREE 
STRONGLY DISAGREE 
STRONGLY AGREE 
AGREE 

NEUTRAL 

DISAGREE 
STRONGLY DISAGREE 


Figure 31 


AFTER-COVID 


PRE-COVID 


m@ Teachers 


13.6% 


BARRIERS TO LEARNING ARE REDUCED IN THE ONLINE 
CLASSES BY INCLUDING CLEAR EXPECTATIONS OF STUDENT 
ACTIVITY (PARTICIPATION AND PERFORMANCE). 
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Recommendation: Ensure that online courses include clearly defined student activity 
expectations, and that these expectations are communicated to students. 


Target Teachers 

Audience 

Needs Expectations for student activity are clearly defined and communicated to students 
Initiatives Develop, define, and communicate expectations for student activity 

Goals To increase presence, communication, and awareness of student activity expectations to 


support successful student learning 
Actions e Develop and define student activity expectations 
e Communicate expectations to students 
CSFs e Student activity expectations are present in the syllabus 
e Student activity expectations are communicated to students 
KPIs e «Quality course review 
e Student activity expectations 


Subdimension: Outcomes 


While the delivery modality may differ in an online environment, the outcomes should be comparable to the 
outcomes for onsite classes. As shown in Figure 33, this was not always the case in online courses before COVID-19 
or after COVID-19. There was a slight increase after COVID 19 (51.3% Administrators; 55.4% Staff; 49.9% Teachers) 
from prior to COVID-19 (45.4% Administrators; 52.6% Staff; 45.6% Teachers). 
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ONLINE CLASSES HAVE COMPARABLE OUTCOMES TO ONSITE 
CLASSES. 


mAdministrators mStaff mTeachers 


STRONGLY AGREE CERETEDERED 
= TA 80.3% 0% a5 
C NEUTRAL 17.5% 19.0% 19.7% 
- DISAGREE { 19.6% 15.5% 18.2% 
< STRONGLY DISAGREE | ERGAEUON: Seas 

STRONGLY AGREE | UBmxQmeeBeycSirleys 
~ AGREE 38.0% 38.0% 
= NEUTRAL 19.0% 19.2% 20.7% 
ec DISAGREE 23.2% 18.0% 21.1% 

STRONGLY DISAGREE | MBRERZCRIO Pa 
Figure 33 


Comparatively, there was a stronger congruence (shown in Figure 34) when teachers were asked the survey 
question “Online classes are designed with specific measurable learning objectives and aligned with assessment 
and learning activities.” These responses reflected 59.6% agreement prior to COVID-19 and 64.0% after COVID-19 
and were significantly higher than teacher responses regarding outcomes being comparable online to onsite 


classes. 
ONLINE CLASSES ARE DESIGNED WITH SPECIFIC MEASURABLE 
LEARNING OBJECTIVES AND ALIGNED WITH ASSESSMENTS 
AND LEARNING ACTIVITIES. 
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Results from these survey questions suggest that outcomes are being used and aligned with classroom activities, 
but the outcomes are not being designed to be comparable to onsite classes. This indicates an area that could be 
improved for future success in online learning. 


52 es 


Recommendation: Measure, review, and communicate student learning outcomes for 
online courses, and if appropriate and/or useful, include communicating comparisons to 
student learning outcomes for onsite courses. 


Target 
Audience 
Needs 


Initiatives 


Goals 


Actions 


CSFs 


KPIs 


Administrators, Staff, Teachers 


Measure, review, and communicate student learning outcomes in online courses, and provide 
comparative onsite data as appropriate 

e Develop and implement student learning outcome metrics for online courses 

e Communicate findings, including onsite comparisons as appropriate 
To develop and communicate an understanding of student learning outcomes in online courses, 
including comparative onsite data as appropriate, to aid in both continual course improvements 
as well as to provide accurate perceptions of the efficacy of online courses 

e Develop student learning outcome metrics for online courses 

e Measure student learning outcomes in online courses 

e Communicate findings, including comparative onsite data as appropriate 

e Use findings to aid in continual improvement of courses 

e Student learning outcome metrics are developed and implemented for online courses 

e Results of these metrics and comparative data are communicated 

e Findings are utilized in continual improvement processes 

e Student learning outcomes 

e Frequency of communication 

e Percentage of administrators, staff, and teachers that indicate that they understand 

student learning outcomes data for online courses 
e Administrator, staff, and teacher perceptions regarding the efficacy of online courses 


Expert Recommendation: Provide rubrics for all assignments. 


Target 
Audience 
_ Needs 
Initiatives 


Teachers 


| Rubrics for all students learning activities and assignments 


Clearly articulate criteria for assignments 


Goals 
Actions 

_CSFs 
KPIs 


____ To provide students with clearly defined criteria for the grading of all assignments 


Develop rubrics linked to student learning outcomes to show grading criteria 


Increased awareness of grading criteria 


Percentage of assignments and activities that have developed rubrics 
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Subdimension: Course Interaction 


In any online class, it is expected that different types of interactions will occur between the students and the 
teachers. The survey asked multiple questions to understand whether interactions were occurring in the online 
environment. This included identifying if student-to-student and teacher-to-student interactions are encouraged. 
The responses reflected that administrators (77.0%), staff (77.8%), and teachers (74.9%) all believed that these 
were occurring before COVID-19. In the responses after COVID-19, there were slight drops in these numbers for 
administrators (73.4%), staff (74.1%), and teachers (69.8%) which may be a result of feedback heard in the 
interviews that teachers did not always feel like they could communicate synchronously with students during the 
transition to remote learning (Figure 35). 


STUDENT-TO-STUDENT AND TEACHER-TO-STUDENT 
INTERACTIONS ARE ESSENTIAL CHARACTERISTICS THAT ARE 
ENCOURAGED AND FACILITATED. 
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This decrease after COVID-19 was also seen in regards to teacher responses to a question about student 
interactions being facilitated in a variety of ways. Before COVID-19, teachers indicated 67.7% agreement with this 
statement but after COVID-19, this decreased slightly to 65.0% agreement. While this was not a significant 
decrease, it may once again be a result of teachers not feeling like they had the same level of communications 
with students during the transition to remote learning (Figure 36). 
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STUDENT INTERACTIONS WITH OTHER STUDENTS AND WITH 
THE INSTRUCTOR ARE FACILITATED IN A VARIETY OF WAYS. 
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This concern about a decrease in engagement opportunities with students continues to manifest itself in other 


questions about interactions. When teachers were asked to rate their agreement regarding students participating 
in collaborative activities (Figure 37), prior to COVID-19 there was higher agreement (70.7%) than after COVID-19 


(63.2%). A similarly significant decrease (Figure 37) is seen when reviewing teacher responses pre and post-COVID- 
19 regarding opportunities for peer interactions. In this case, the level of agreement before COVID-19 was 72.0% 
and dropped to 64.3% after COVID-19. 
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STUDENTS PARTICIPATE IN MEANINGFUL WORK GROUPS 
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PROMOTING COLLABORATION. 
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PEER INTERACTION FACILITATES ACTIVE LEARNING THROUGH 
FREQUENT AND ONGOING PEER-INVOLVEMENT. 
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Figure 38 


Recommendation: Encourage opportunities for teacher-to-student and student-to- 


student interaction in the online classroom. 
Target NELC, Ministry of Education, Administrators, Staff, Teachers 
Audience 
Needs Policies and processes regarding interactions in the online classroom 
Initiatives Support teacher-to-student and student-to-student interactions 
Goals To encourage teacher-to-student and student-to-student interactions in online classes 


Actions e Develop and implement policies and process at the national and local level to 


CSFs 
KPIs 
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encourage interactions in online classes 
e Teachers need to encourage and ensure opportunities for interactions in their 
classes 
Teacher-to-student and student-to-student interactions occur in the online classroom 
e Student satisfaction with interactions 
e Teacher satisfaction with interactions 


Expert Recommendation: Establish the presence of the online educator. 


Target Teachers 

Audience 

Needs _ Opportunities for the student to get to know the teacher 
Initiatives Establish online presence of the educator 

Goals Provide opportunities for the student to better know their teacher 
Actions e Include an instructor biography and contact information 


e Share expertise with students through external resources (especially those 
written/prepared by the educator, such as teacher portfolios) 
e Regularly participate in discussions or other collaborative activities 


CSFs Increased engagement between the teacher and students 


KPIs _ Frequency of teacher contributions in the virtual classroom 


Subdimension: Feedback 


When students do not have face-to-face access to an instructor, they rely on timely and substantive feedback to 
keep track of their progress and success in the online course. This fact appears to be understood by those in the K- 
12 schools since most survey respondents were in agreement. However, the responses appeared to decline from 
the pre-COVID-19 (72.7% Administrators; 73.2% Staff; 76.0% Teachers) results to after COVID-19 (70.0% 
Administrators; 71.1% Staff; 72.2% Teachers) (Figure 39). 


TEACHERS PROVIDE TIMELY AND DETAILED FEEDBACK ON 
ASSESSMENTS AND STUDENT INQUIRIES. 
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The feedback that students receive also needs to include adequate instruction on how to improve. When teachers 
were surveyed on providing feedback that is constructive and timely, pre-COVID-19, 74.4% indicated that this was 
occurring. However, this declined after COVID-19 to 70.0%. While this does not represent a huge change, the 
responses to both survey questions related to Feedback, reflect that a change occurred during COVID-19. It is 
likely that teachers had difficulty keeping up with expectations for feedback and meeting students needs due to 
the rapid transition to remote learning. 


Recommendation: Set expectations for feedback provision and methods, as well as when 
feedback will be provided and how it is related to course learning objectives. 


Target Teachers 

Audience 

Needs Feedback provision and methods, timing, and connection to learning objectives is present and 
communicated to students 

Initiatives Develop and communicate methods for providing feedback, and set expectations for timing 
and explanations of how feedback is related to learning objectives 

Goals Support successful student learning outcomes by establishing and communicating clear 
processes and expectation for feedback 

Actions e Develop processes and methods of feedback provision in alignment with course 

activities 


e Communicate feedback processes and methods, and set expectations including 
timing and how feedback is connected to learning objectives, to students 
CSFs e Feedback processes and methods are developed and implemented 
e Communication to students regarding feedback processes, methods, connection to 
learning objectives, and expectations is provided 
KPIs e Percentage of teachers who implement processes and methods of feedback 
e Frequency of communication to students regarding feedback 
e Percentage of students reporting that feedback processes, methods, and provision are 
implemented, understood, and support learning 
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Subdimension: Innovation 


When supporting online students, there also needs to be opportunities for innovation. This is how we as educators 
learn and improve our methods. It was interesting to note that administrators (67.9%), staff (69.3%), and teachers 


(62.5%) indicated agreement prior to COVID-19 that innovation was encouraged. After COVID-19, the responses 


reflected that innovation in online teaching and supporting students was even more strongly encouraged (74.4% 


Administrators; 73.6% Staff; 67.4% Teachers). These results reflect a strong willingness by teachers and staff to 
engage in innovation with more support from administrators (Figure 40). 


TEACHERS AND STAFF ARE ENCOURAGED TO INNOVATE IN 
ONLINE TEACHING AND IN THEIR SUPPORT OF STUDENTS. 
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Figure 40 
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Recommendation: Consistently provide and encourage opportunities for innovation in 
online teaching and support of students. 


Target Administrators, Staff, Teachers 
Audience 
Needs e Encourage the discovery, dissemination, and implementation of innovation 


e Provide professional development on the design, development, and delivery of 
innovative teaching and student support 
Initiatives e Explicitly encourage innovation through communications, activities, and professional 
development 
e Develop and implement professional development opportunities on innovative 
teaching and student support 
Goals To increase the design, development, and delivery of innovative teaching and student support 
Actions e Develop and implement strategies for the discovery and dissemination of innovative 
teaching and student support techniques 
e Develop and implement professional development opportunities for teachers and staff 
on the design, development, and implementation of innovative teaching and student 
support 
CSFs e Innovation actively encouraged 
e Professional development opportunities related to innovative teaching and student 
support are provided 
KPIs e Frequency of communications to teachers and staff encouraging innovation 
e Teacher and staff participation in professional development opportunities 
e Percentage of teachers and staff that implement and measure innovative practices 


Dimension IV: Assessment 


The Assessment dimension measures four subdimensions including Assessment Strategies, Assessment 
Processes, Assessment Methodology, and Innovation. 


Findings related to readiness for assessment in a fully-online environment indicate overall agreement on 
subdimension items, though at a lower rate and with more neutral responses than some others. Survey and 
interview results show that there is agreement with regard to the communication, understanding, and 
implementation of online assessment. However, teachers were not prepared to complete course assessments in 
this environment, and results further suggest that in order to ensure readiness for effective assessment practices 
online, clear expectations and agreement should be established regarding strategies, processes, and 
methodology to adequately measure learning. 


Overall findings in the Assessment dimension indicate that school administrators, staff and teachers are 
consistently in alignment with regard to their opinions, perceptions and anticipations across all subdimensions, 
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suggesting that their interactions regarding assessment and the communication of assessment practices are quite 
effective among each other. In general, school administration, staff and teachers agree at rates of around 60% or 
remain neutral at approximately 20% that the queried practices in each subdimension are present. Notably, the 
percentage rates in the Assessment dimension are somewhat or sufficiently close when considering school 
administrators’ and staff responses as compared to teachers’ replies on the same items, and subdimensions. 


The consistency in response among school administrators, staff and teachers as a group lends strength to the 
primary finding that there are no significant gaps in communication, understanding, and implementation of 
assessment strategies, processes, methodologies, and types of assessments between the group of administrators, 
staff and teachers. These findings are described below by subdimension. 


Measures of satisfaction with items, dimensions, and subdimensions prior to and after COVID-19 among the target 
groups reflect that the majority of participants are overall satisfied or prefer to express neutral attitude towards 
queried items of the research. It is noteworthy, that both satisfied and very satisfied responses during the pre- 
pandemic period decrease, though slightly, in the post-COVID phase, similar to unsatisfied and very unsatisfied 
responses that also show an identifiable downtrend through the pandemic. Meanwhile, neutral statements 
become more popular among respondents and their numbers increase in the after-COVID time. 


Interview responses related to the assessment dimension overall supported the findings from the survey items on 
assessment. Selected interview responses are developed into the subdimension results below, however, it is 
worth emphasizing that several themes emerged in the qualitative interviews that were not queried in the survey 
itself. First, some teachers and administrators indicated concerns about academic integrity in online assessments 
- in other words, ensuring that the student enrolled was the individual completing assessments, as well as that 
the assessments accurately measure the anticipated learning outcomes. Second, a number of teachers pointed 
out that they strongly advocate for online assessment procedures, however some school principals may not grant 
approval to e-assessments, particularly e-exams, considering them informal. Taking into consideration the 
unprecedented education crisis and the urgency of measures taken, relative authorities encouraged schools to 
perform evaluation of students during COVID-19 from the perspective of their results in the previous semester. 
Finally, the post-pandemic means for improving assessment process, from teachers’ perspective, is to establish 
evaluation committees that would be responsible for developing online exams. 


Subdimension: Assessment Strategies 


Measures of satisfaction with assessment strategies before and after COVID-19 among school teachers indicate 
that the greater share of participants are overall satisfied or prefer to keep neutrality while answering survey 
questions. The point to be emphasized is that the satisfied responses decreased (total decline by over 3%) in the 
post-COVID phase, while very satisfied, generally unsatisfied and very unsatisfied replies showed a moderate 
upturn through COVID. With an increase of 1.5% in the after-pandemic period, the neutral responses remained 
second most popular among the target groups (Figure 41). 
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Figure 41 


AFTER- 


COVID 


PRE-COVID 


ONLINE CLASSES HAVE MULTIPLE TIMELY AND APPROPRIATE 
ACTIVITIES TO ASSESS STUDENT LEARNING OF THE ALIGNED 
LEARNING OBJECTIVES THROUGHOUT THE CLASS. 
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School administrators, staff and teachers - all responded in alignment with the consistent pattern noted above. 
Qualitative interview responses further supported these findings, and respondents provided discussion not only of 
readiness, particularly in the remote instruction environment, but also gave clear examples of piloted assessment 
strategies to address learning needs, as well as noted a diverse set of formative assessment strategies. By far, the 
majority of respondents support e-assessment and evaluation of student learning outcomes and subject 
proficiency, which is evidenced by over 50% agreement replies on the existence and actualization of ongoing 
assessment strategies to measure content knowledge, skills, and abilities while providing students feedback on 
what they have learned and on what to improve (Figure 42). 
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PRE-COVID 


ONGOING ASSESSMENT STRATEGIES ARE USED TO MEASURE 
CONTENT KNOWLEDGE, SKILLS, AND ABILITIES WHILE PROVIDING 
STUDENTS FEEDBACK ON WHAT THEY HAVE LEARNED AND ON 
WHAT TO IMPROVE. 
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Figure 42 

Recommendation: It could be presumed, that post-COVID responses from educators on 
satisfaction with the present assessment strategies, procedures and techniques would 
change somehow or even considerably in comparison with the pre-pandemic times. Since 
alignment between learning objectives, instructional activities, and assessments is 
equally critical for traditional and online learning, it is essential to reduce the existing 
gaps between school administrators, staff, teachers, and students in communication, 
expectations, and implementation of assessment strategies, processes, methodologies, 
as well as types of assessment. 


Target Administrators, Teachers, Students 

Audience 

Needs Provide insightful information on assessments, their objectives and overall learning outcomes 
to students 


Initiatives Elaborate and implement subject-related communications to students regarding assessments, 
their objectives and overall learning outcomes 

Goals To ensure student understanding and appreciation of assessment activities to increase learning 
efficiency and promote successful educational outcomes 

Actions Develop and provide communications to students via various media (syllabi, webinars, email, 
LMS communications, fora, etc.), clearly identifying assessment strategies, processes, 
methodologies, types of assessments, their objectives and overall learning outcomes 

CSFs Schools provide insightful and accurate information about the existing and/or developed 
assessment types and procedures, the rationale behind these strategies and their clearly- 
worded objectives 

KPIs Frequency of communication to students for comprehensive and up-to-date information on 
ongoing assessment strategies (formative and summative assessment), their objectives, overall 
learning outcomes and educational impact 


Subdimension: Assessment Processes 


Measures of satisfaction with assessment processes both prior to and after COVID-19 among school 
administrators, staff and teachers highlight that the majority of participants are overall satisfied or stay neutral on 
the research items. It is of note, that both satisfied and very satisfied responses decreased post-COVID while 
neutral, unsatisfied, and very unsatisfied responses increased through the pandemic. 


Assessment processes in this study included alignment of learning objectives and assessments, effective 
communication of assessment criteria, grading, expectations, and feedback. Findings in this subdimension 
followed the clear pattern of responses, with the majority of teachers (from 54 up to 58% positive replies) 
indicating that each of these items is provided at high rates and featuring a strong and vivid percentage-point 
difference with strong disagreement responses (~3% negation that these elements are present and/or effective). 
Qualitative interviews did illustrate teachers’ awareness of the existing complexities and challenges with 
implementation and student understanding of assessment procedures due to the pandemic. 
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Recommendation: Successful education systems stipulate both high quality and equity, 
supporting all students to achieve personal self-fulfillment. A well-developed assessment 
system can encourage learning and ensure that policies and practices promote equally 
high standards of education for all children, regardless of their context, circumstances or 
location. An open dialogue, reflection and feedback, where shortcomings and mistakes 
are acknowledged and recognized by all stakeholders, evaluation and assessment will 
contribute vastly to building trust between all members of the educational process. 
Ensure alignment of students’ learning objectives and assessments through transparent 
quality review processes. 


Target Administrators, Teachers 
Audience 
Needs Ensure alignment of learning objectives and assessments 


Initiatives Promote open dialogue with all members of the educational process 
Goals To support successful student learning through setting out clear and direct objectives for what 
is expected nationally of students, teachers, schools and the entire K-12 system where 
assessment serves to advance teaching and learning 
Actions e Align assessment with national goals for student learning 
e Create constructive interaction between school administration, teachers and students 
to help students progress and be even better motivated to learn and achieve 
CSFs Assessments and reporting of results are aligned with the learning standards, so that teachers, 
students, school administration and parents have a clear understanding of students’ 
advancement and/or of any gaps and difficulties in their learning 
KPIs e Student learning outcomes met 
e Alignment of learning objectives with assessments 
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Recommendation: Clearly describe assessment criteria when assignments are issued, 
provide grading rubrics and samples of assignments illustrating teacher/ school/ system 
expectations. 


Target Administrators, Teachers 
Audience 
Needs Align subject materials, including criteria, assignments, expectations, and grading rubrics with 


best practices 
Initiatives Ensure that assessments pursue generally accepted best practices and directly correlate to 
teacher expectations 
Goals To support student excellence in learning outcomes through the use of effective course design 
and assessment for a more student-focused, competency-based education 
Actions e Align subject-related assessments with adequate course design, including delivery 
and placement 
e Ensure that assignments, criteria, expectations, and grading rubrics are present and 
that these elements are timely and clearly communicated to students 
CSFs e Subject-related assessment delivery and placement follow the established course 
design standards 
e Assignments, criteria, expectations, and grading rubrics are present and are timely 
and clearly communicated to students 
KPIs e «Quality reviews 
e Number of communications concerning assignments, criteria, expectations, and 
grading rubrics meaningfully provided to students 
e Student learning outcomes met 
e Course evaluations and feedback 


Subdimension: Assessment Methods 


Measures of satisfaction with assessment methods before COVID-19 among school administrators, staff and 
teachers reflect that the majority of participants are overall satisfied or very satisfied. In the post-pandemic 
period, the tendency shifted from strong agreement toward neutrality with general satisfaction keeping the lead 
in the response rates. It is worth emphasizing that both satisfied and very satisfied responses decreased post- 
COVID, while unsatisfied and very unsatisfied responses increased. This dynamic is particularly peculiar to the 
teacher group of respondents with general disagreement about the queried items increasing by over 2% (upward 
of 1,000 replies) through the pandemic (Figure 43). 


| OLIA Rana 


TEACHERS USE AN ACTIVE LEARNING ASSESSMENT APPROACH 
WHERE STUDENTS ARE NOT ASSESSED SOLELY ON TESTS/QUIZZES 
BUT ARE PROVIDED AMPLE OPPORTUNITY TO DEMONSTRATE 
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Figure 43 


PROFICIENCY IN DIFFERENT WAYS. 
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Additionally, in this subdimension, school administrators, staff and teachers were required to respond to survey 


items regarding peer- and self-assessment, as well as feedback opportunities. Qualitative interview responses 


indicated that teachers are enthusiastic and highly motivated to introduce and practice various types of 


assessment, which is evidenced by their own and students’ efforts for collaboration, teamwork, peer review and 


self-assessments. However, challenges emerge when teachers are asked about opportunities to organize 


diversified assessments of student progress and set goals for students’ personal and educational improvement. 


These complications concern the standing of educational authorities and supervisors about the credibility of peer- 


and self-assessment. Qualitative interviews exposed the effective and engaging practice of holding student 


competitions to evaluate subject proficiency, along with each student’s mastery of the curriculum and level of 


creativity and motivation. 
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Recommendation: Provide students opportunities to demonstrate their subject 
proficiency in various ways. Diversify formative and summative assessment. 


Target 
Audience 
Needs 


Initiatives 
Goals 


Actions 


CSFs 


KPIs 


Teachers 


Ensure a greater variety of assessment methods to enable most accurate student subject 


proficiency and individual student’s mastery of the curriculum in response to instruction and 


self-education 


Use diverse assessment approaches and techniques to better meet various students’ needs 


To raise levels of student success and achieve greater equity of learning outcomes through 


more frequent, transparent, engaging and interactive assessments 


Develop, gain approval and implement engaging assessment methods for students (peer- and 


self-assessment) 


Peer- and self-assessment methods are developed and used on a regular basis 
Students gain diversified opportunities to display their subject proficiency and 
individual mastery of the curriculum 

Quality ratings of student learning outcomes 

Frequency of transparent, engaging, and interactive assessments 


Subdimension: Innovation 


Measures of satisfaction with assessment innovation both pre- and post-COVID among school administrators, staff 


and teachers reflect that the majority of participants are generally satisfied or keep themselves neutral. It should 
be, nonetheless, highlighted that very satisfied responses increased post-COVID among school administration, 
staff and teacher groups (by total upward adjustment of 1%, representing 500 respondents), however, the general 
satisfaction rates decreased from pre- to post-pandemic phases. The overall dissatisfaction about the relative 


survey items experienced some fluctuations that resulted in a very slight rise (by 0.2%) of the strong disagreement 
rate and just as small decline in general dissatisfaction index (by 0.3%) (Figure 44). 
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TEACHERS PILOT ASSESSMENT STRATEGIES THAT INCORPORATE 
DIGITAL LITERACIES AND OTHER PERTINENT LITERACIES. 
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Figure 44 
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The innovation subdimension for assessment was measured through queries on new strategies and the use of 


new pedagogical methods and opportunities to demonstrate learning in a variety of ways. Teachers were most 


enthusiastic in their responses to point out their support of integrating advanced technology, platforms, 


instruments and tools. Qualitative data emphasizes teachers’ high interest in innovative assessments for online 


courses, both in their current practice and prospective educational activities. 


The recommendations below relate to innovation, the development, dissemination and implementation of 
innovative assessment strategies, processes, methods, and types. Further, school administration, staff and 
teachers should be directly encouraged to include innovative assessments in their online courses, and provided 
pedagogues with sufficient training on conceptualization, design and delivery of these innovations. 
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Recommendation: Innovation in assessment strategies, processes, methods, and types is 
encouraged, supported, and actualized; adequate and consistent professional 
development on design, development, and delivery is provided. 


Target Administrators, Teachers 
Audience 
Needs e Development, dissemination, and implementation of innovative (ICT-based) 


assessment strategies, processes, methods, and types 
e Provide professional development on the design, development, and delivery of 
innovative assessment 
Initiatives e Encourage innovative assessment through communications, activities, and related 
professional development 
e Develop and implement professional development opportunities on innovative (ICT- 
based) assessment 


Goals To beneficiate teacher-student interaction and enhance the ways in which students receive 
evaluations and engage with feedback 
Actions e Develop and implement strategies for the discovery and dissemination of innovative 


assessment techniques 
e Design various ICT-based assessment components 
e Conduct follow-up on the use of innovative assessment activities to ensure timely 
improvements and avoid confusion 
e Develop and implement professional development opportunities for teachers on the 
design, development, and implementation of innovative assessment methods 
CSFs e Innovation relevant to assessment is actively encouraged and supported 
e Various ICT-based assessment components developed and implemented; student 
responses obtained 
e Professional development opportunities related to innovative assessment are 
provided 
KPIs e Number and clarity of communications 
e Participation in professional development opportunities related to innovative 
assessment methods 
e Percentage of teachers who implement and measure innovative assessment practices 
in their practice 


Dimension V: Technology 


The Technology dimension measures nine subdimensions including Centralized Online Education Infrastructure, 
Operability, Modality, Security, ITSM Compliance, Internet Access, Reliability, Coverage, and Innovation. 


Prior to COVID-19, there were clear needs related to technology that were planned for in long-range online 
education plans. Post-COVID, demands on technology increased, and survey and interview results that although 
the areas represented in the technology dimension were not fully prepared for the shift, improvements were and 
continue to be planned and implemented. Of particular note in this subdimension, increasing the capability of 
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centralized infrastructure to manage high-volume use, ensuring security related to student data and privacy, and 
making improvements to internet access, reliability, and coverage, including both network and device challenges, 
are all needed to ensure readiness in upcoming terms. 


Overall quantitative and qualitative findings indicate that technology improvements are recommended. Data 
indicate that the primary foci for technology reflect needs related to internet access, reliability, coverage, and 
operability. Recommendations for this dimension largely focus on the development of processes to monitor and 
assess technology in order to better identify associated challenges and needs, and to implement solutions as 
appropriate to increase users’ ability to effectively participate in online learning. The need for these 
recommendations is further confirmed in the OECD report How the COVID-19 Pandemic is Changing Education: A 
Perspective from Saudi Arabia, which indicates that many students may lack the resources needed for remote 
learning. 


Subdimension: Centralized Online Education 
Infrastructure 


Administrators, staff, and teachers expressed consistent perceptions of measures assessing centralized online 
education infrastructure, including the presence of a centralized learning management system (LMS), technology 
plans, single-sign-on (SSO), and support. Those agreeing or strongly agreeing that these elements are present 
ranged from 54.6-70.3% and disagreement or strong disagreement ranged from 10-19.6%. These figures are stable 
both pre-and post-COVID. Overall, agreement responses increased post-COVID, while disagreement responses 
decreased. Most notably, findings indicate that SSO capabilities reported by administrators, staff, and teachers 
increased post-COVID. Responses from parents and students regarding centralization and SSO indicate a similar 
trend, though disagreement or strong disagreement increased by nearly 10 percentage points post-COVID (Figures 
45, 46). 
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Figure 46 
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Qualitative data indicate that technology was present pre-COVID, but not at the centralized level necessary to 


adequately support online learning. One administrator articulated this challenge: “Although the school was up-to- 


date in technology, we were not 100% ready to offer fully online courses and distance learning methods, which is 


something we had to do and work with during the pandemic. Before the pandemic, technology was blended in the 


regular classrooms, but having 100% online courses for all of the grades was something new on the school.” 


ef) 
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Recommendation: Provide a centralized learning system with single sign-on (SSO) or 
single identification. Focus should prioritize schools that have not yet implemented 
online learning as supported by the Future Gate initiative, as these are the most likely to 
indicate absence of centralized solutions. 


Target Ministry of Education 

Audience 

Needs Increase access to a centralized online learning program 

Initiatives Identify schools that have not yet implemented online learning and provide a centralized 
LMS with SSO 

Goals To increase the number of schools successfully implementing online learning 

Actions e Identify schools who do not have an LMS 


Help schools determine any contextual needs 
Ensure interoperability standards between the LMS and other systems 
Implement SSO 


CSFs e Schools with needs identified and contextualized 
e LMS and SSO adopted and implemented 
e Students successfully log into LMS 

KPIs Percentage of schools with an LMS and SSO 


Subdimension: Operability 


When asked about the operability, reliability, and consistent access to online courses, administrators, staff, and 
teachers were closely aligned in their perceptions both pre- and post-COVID, ranging from 66.1-69.4% agreeing or 
strongly agreeing and 8.8-11.7% disagreeing or strongly disagreeing. Parent and student responses were also 
consistent both pre-and post-COVID, though at lower rates (50.1-53% agreed or strongly agreed; 25.7-31.8% 
disagreed or strongly disagreed). 
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Recommendation: Given the disconnect between the educators and parents and 
students, schools can develop feedback channels and provide orientation and training 
around learning technologies. 


Target Administrators, Staff, Parents, Students 

Audience 

Needs Improve the ability for students and parents to use learning technologies 
Initiatives e Develop a feedback loop to ensure parent and student needs are met 


e Create centralized resources that are comprehensible and policy accessible for 
parents and students 
Goals To improve understanding about using learning technologies 
Actions e Create a mechanism for parents and students to ask questions about learning 
technologies 
e Curate relevant resources related to challenges 
e Develop an open, centralized space for parents and students to share common pain 
points for the school and access resources 
CSFs e Parents and students share challenges 
e Students have access to technology 
e Curated resources are easy to find and access 
KPIs e Percentage of students who can use learning technologies 
e Website visits, page views, and hits 


Subdimension: Modality 


Administrators, staff, and teachers responded to questions assessing two types of modality: asynchronous or 


synchronous delivery, blended learning opportunities. Responses among all three groups indicated improvement 


post-COVID, with the largest increase in agreement or strong agreement present among administrators when 
asked about synchronous and asynchronous options (Figure 47). 
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ONLINE CLASSES ARE DELIVERED ASYNCHRONOUSLY AND 
SYNCHRONOUSLY DEPENDENT ON THE LEARNING OBJECTIVES. 


mAdministrators mStaff mTeachers 


STRONGLY AGREE _ | BRABYSRPA CERES 


AFTER-COVID 


STRONGLY AGREE | BESEAHRORSY Press 


PRE-COVID 


AGREE 55.6% EVA 54.5% 


NEUTRAL _ 23.3% 22.0% 24.9% 


f 
DISAGREE | Beksleec¥as 


STRONGLY DISAGREE 


7 


AGREE 52.7% 56.9% 54.0% 


NEUTRAL 25.7% 22.9% 26.4% 


DISAGREE | SBexsSRax. cys 


{ 
STRONGLY DISAGREE _ Seeakay7 


Figure 47 


Qualitative interview responses underscore the need for effective use of modality in the provision of online 
courses. One teacher elaborated on these needs, saying “The most important thing to support e-learning is to 
provide computers and internet service. These are the most important two things. Many students call me telling 
that they don’t have a device or they just have one device at home. As you know, Saudi families are big families, 
where the number of family members in a household usually starts with 7 and increases. So, when a student’s 


family has only one computer, using it is hard. Same with the teacher.” 


Expert Recommendation: Evaluate the student experience in synchronous and 
asynchronous online learning in order to improve course design and outcomes. 


Target Teachers, Students 
Audience 
Needs Understand the student experience in online learning to improve courses 
Initiatives e Evaluate student experience over time 

e Reflect on course design and areas for improvement based on needs 
Goals To better support students and improve learning outcomes 
Actions e Collect feedback from students about asynchronous and synchronous learning 

experiences 

e Evaluate feedback and use to inform appropriate course changes 
CSFs e Students sharing feedback 

e Feedback is implemented into course design 
KPIs e Student outcomes 
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| e Frequency of feedback collection 


Subdimension: Security 


Administrators and students report consistent responses regarding data security measures, including the integrity 
and validity of information, identity access, and secure links and information in online courses. Rates of 
agreement or strong agreement range from 63.3-71.7%, while disagreement or strong disagreement indicate 6.9- 
17.8%. Parents indicated that they were less sure of course security, though the differences do not have a large 
range. Parents agreed or strongly agreed at 58.8-64.3%, and disagreed or strongly disagreed at 14.4-22.2%. The 
highest area of concern indicated by parents was related to the security of students’ identity and information, in 
contrast with administrators (Figures 48, 49). 


DIGITAL SECURITY MEASURES ARE IN PLACE TO ENSURE THE 
INTEGRITY AND VALIDITY OF INFORMATION, INCLUDING IDENTITY 
ACCESS. 
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| FELT MY CHILD'S IDENTITY AND INFORMATION IN THE CLASS 
SITES WERE SECURE. 
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Figure 49 


Security for students is a concern that schools indicate awareness of. In one administrator interview, the 
participant noted not just a focus on security for students, but for teaching students to understand internet 
security and to protect themselves: “Yes, the first thing we started to do is that we gave an integrated digital 
program on how to the students can protect themselves against harmful websites even before using the Internet 
and the phone, and how they should not open any link or see any clip, and we were the only school to implement 
this program.” 


Recommendation: As schools increasingly make use of learning technologies, ensuring 
privacy and data security become greater risks. Data breaches have become increasingly 
common in schools and stakeholders often do not understand terms of service and are not 
provided with details about how data is collected, stored, and used. Greater transparency 
and clear policies and guidelines can increase stakeholder trust and buy-in. 


Target Administrators, Staff, Teachers, Parents, Students 
Audience 
Needs e Increase trust and confidence in for data security 
e Clear policies and practices to ensure data security 
Initiatives e Develop aclear data security plan and share with stakeholders 
e Develop policies around the use of third-party applications 
Goals To decrease privacy and security risks 
Actions e Educate practitioners and families about the risks of using third-party applications 


that have not been appropriately vetted 
e Create a process for vetting applications and share with practitioners 
e Develop guidelines for student data use and share with stakeholders 
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CSFs e Practitioners avoiding potentially malicious applications 
e Stakeholders participating in conversations 
e Sufficient expertise to vet technologies and making use of ministry-provided 
websites and applications 
KPIs e Number of applications in schools 
e Reported trust 


Subdimension: Information Technology Service 
Management (ITSM) Compliance 


It is unlikely that those not directly involved in ITSM compliance (including staff, teachers, parents, and students 


would be able to respond to this item. For that reason, administrators were the only stakeholder group queried, 
and indicated that pre-COVID, 64.0% agreed or strongly agreed that ITSM designed, delivered, operated, and 
controlled information supporting online course delivery and development and 9.5% disagreed or strongly 
disagreed. After COVID, agreement or strong agreement increased to 69.2% and disagreement or strong 
disagreement decreased to 7.2%. 


Recommendation: While the results were generally positive among administrators, a 
large percentage did not feel that ITSM satisfactorily supported online course delivery. 


Administrators should evaluate current processes, plans, and resources that are available 


and where gaps or inefficiencies exist. 


Target Administrators, ITSM 

Audience 

Needs Evaluate current processes, plan, and resources to ensure successful online course delivery 
Initiatives e Determine IT pain points among stakeholders 


Goals 
Actions 


CSFs 


KPIs 


Evaluate current processes, plans and resources 
Develop new resources to address pain points 


To increase support for online course delivery 


Communicate with different stakeholder groups (staff, students, teachers, parents) to 
determine pain points (survey, focus group, town hall, etc.) 

Use identified pain points to examine current internal and outward-facing resources 
and address gaps or areas to improve 

Create new resources and share in a manner that is easy to find 

Various stakeholders engaged 

Pain points determined and addressed 

Number of support tickets/requests 

Number of resources accessed 

Stakeholder satisfaction 


Subdimension: Internet Access 


TT 
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Regarding internet access from school, parents and students reported pre- and post-COVID rates of agreement 
between 34.5-38.1%, with disagreement or strong disagreement at 39-41.6%. Internet access from home is 
reported at higher rates (55.4-63.4%) and disagreement or strong disagreement is lower (22.4-30.7%). These 
findings indicate the consistent need for better internet access both at school and home, and administrator, staff, 
and teacher responses support this recommendation (Figures 50, 51). 


THE INTERNET IS SUFFICIENTLY AVAILABLE IN THE SCHOOL 
ENVIRONMENT. 
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WE HAVE INTERNET ACCESS IN OUR HOME THAT IS AVAILABLE 
WHEN IT IS NEEDED. 
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Qualitative interviews reveal that internet access is a top challenge for all stakeholder groups. This includes both 
access to sufficient internet services as well as access to devices such as computers, tablets, or smartphones for 
course access. Interview data indicated challenges with no internet or weak internet at both schools and home, as 
well as that not all students, or even all teachers are able to access devices reliably. Interviews also indicate that 
these challenges are pronounced for public schools and students, whereas private schools and students may have 
more resources. 


Recommendation: For students who have poor or no internet access at home, schools can 
provide mobile hotspots and develop content that requires minimum bandwidth. 


Target Administrators, Staff, Teachers, Students 

Audience 

Needs Determine and address student internet access needs 
Initiatives e Identify student internet needs 


e Develop a distribution plan to share mobile hotspots 
e Evaluate course content to ensure options for students to continue work with poor 


internet 
Goals To maintain continuity of learning to the greatest possible extent 
Actions e Communicate with families through different channels to determine need 


e Procure and distribute mobile hotspots with a guide for use 
e Maintain a feedback loop for future internet access needs 
e Teachers evaluate content to minimize the amount of synchronous online sessions, 
high resolution recordings and images, etc. 
e Teachers develop print contingency materials for students to complete in the event 
of a sudden, sustained loss of internet 
CSFs e All students have access to internet to complete their work 
e Families can report loss of internet and receive technology after start of the school 
year 
e Teachers trained on how to vet high-bandwidth content 
KPIs e Percentage of students who have access to internet 
e Number of hotspots distributed 
e Percentage of students who continue to make satisfactory academic progress 
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Subdimensions: Reliability 


Interestingly, the reliability subdimenson indicates consistent levels of agreement or disagreement regarding the 
ability to reliably log in to core learning technology systems and courses among administrators, staff, teachers, 
and students. There was a clear gap between these groups and the perceptions of parents. Reliability 
improvements are indicated across all groups, though post-COVID, parent responses agreeing or strongly agreeing 
with log-in reliability increased to reflect similar levels to other groups. It is possible that parents were simply 
unaware of reliability issues prior to COVID, when students were not required to remotely access courses as often. 
Due to the increase in demand on reliable systems, it is recommended that the approach to increasing reliability 
include the development and implementation of monitoring processes to better understand challenges, followed 
by processes for systemic improvement (Figure 52). 


MY CHILD/I WAS CONSISTENTLY ABLE TO LOG IN TO ONLINE 
COURSES. 
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Figure 52 


Specifically, regarding reliability, and related to other subdimensions of technology such as internet access and 
centralization, one teacher said of the challenges: “Sometimes it takes more than half an hour for the system to 
open, so the class time finishes before we get to be on the platform. This also happened to us, the high school 
teachers, during the pandemic, we suffered a lot from the lack of the support and the load on the network.” 


Sear 


Expert Recommendation: Monitor the reliability of learning technology systems and 
courses as related to infrastructure (i.e. challenges resulting from infrastructure, not 
from individual internet access, etc.). Identify and address areas needing system-level 
solutions. 


Target Administrators, ITSM 
Audience 
Needs Identify system-level and/or infrastructure challenges related to the reliability of access to 


learning technology systems and courses 


Initiatives Develop processes for monitoring and addressing the reliability of infrastructure and learning 
technology systems and courses 


Goals To increase the awareness and problem-solving of system-level infrastructure reliability as 
related to access of learning technology systems and courses 


Actions e Develop processes for monitoring reliability 
e Develop processes for problem-solving system reliability issues 
e Implement solutions to address reliability challenges 


CSFs e Processes for monitoring reliability are developed and implemented 
e Processes for resolving reliability challenges are developed and implemented 


KPIs e Number of system reliability issues 
e Feedback from stakeholders 
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Subdimension: Coverage 


Administrators, staff, and teachers indicated both pre- and post-COVID that approximately half agreed or strongly 
agreed that internet access and access to core learning technologies are available from any location (46.1-57.6%), 


which they disagreed or strongly disagreed at rates ranging from a quarter to approximately one-third (24.7- 


37.1%). These findings indicate that while consistent coverage is present (and moreso post-COVID), improvements 
are necessary to support sustained online learning (Figure 53). 


INTERNET ACCESS AND ACCESS TO CORE LEARNING TECHNOLOGIES 
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Figure 53 


ARE AVAILABLE FROM ANY LOCATION. 
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In alignment with previous subdimensions, issues of coverage were a theme in the qualitative interviews. Many 
respondents indicated that their schools lack sufficient coverage, as do the areas in which they live. Both of these 
create challenges for effective online learning. 
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Recommendation: Given the need for increased learning technologies coverage, a plan 
that includes monitoring and feedback loops to continually determine need, as well as 
mechanisms to provide access to technologies is critical. 


Target Administrators, Staff, Teachers, Students 
Audience 
Needs Developing mechanisms to determine stakeholder needs and ways to appropriately address 
them 
Initiatives e Stakeholders have multiple ways to report needs 
e Periodic follow-ups and opportunities to acquire new technologies 
Goals To decrease coverage gaps for learning technologies 
Actions e Determine appropriate communication channels for stakeholder groups 


e Communicate via all channels and follow-up with non-respondents 
e Maintain feedback loop to ensure coverage throughout the year 
e Ensure that schools have enough technologies to support student remote/online 
learning 
CSFs e Stakeholders report needs 
e Continued communication and opportunity to determine needs 
e Adequate technologies to support needs 
e Needs are evaluated and technologies are provided 
KPIs e Percentage of stakeholders have appropriate access over time 
e Number of technologies to meet needs 


Subdimension: Innovation 


Administrators, staff, and teachers were asked for their perceptions that “new technologies are piloted, 
supported, and encouraged in online classes.” Over half, 59.4-66.6% agreed both pre- and post-COVID that 
innovation is present and encouraged; less than 20% (13.4-18.8%) disagreed or strongly disagreed. Responses 
indicated slight increases in agreement and decreases in disagreement post-COVID, suggesting that technological 
innovation became more encouraged and supported following the shift to emergency remote teaching (Figure 
54). 


84 erKFKrr_ereeeeeeeee—_OoOoOooESEe —n—n—n—n— e—  kkmIaEaeaeEeeooooeeeeeeeeeeee 


NEW AND INNOVATIVE TECHNOLOGIES ARE PILOTED, SUPPORTED, 
AND ENCOURAGED IN ONLINE CLASSES. 
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Expert Recommendation: Monitor the reliability of learning technology systems and 
courses as related to infrastructure (i.e. challenges resulting from infrastructure, not 
from individual internet access, etc.). Identify and address areas needing system-level 
solutions. 


Target Ministry of Education, NELC, Universities, School Administrators 

Audience 

Needs Increase training on educational data and learning analytics tools and approaches in educator 
preparation programs 

Initiatives Curriculum modification to include gaining experience working with student data at different 
levels of granularity 

Goals To increase data literacy and understanding of analytics to improve student outcomes 

Actions e Work with educator preparation programs to evaluate and revise curriculum for 


developing data literacy and analytics skills 
e Develop and provide sustained training at different experience levels for practitioners 
e Provide time for teachers to work with their students’ learning data 


CSFs e Educator preparation programs and curricula are revised to include skill 
development over the course of the program 

e Training is sustained and builds skills over time instead of one-off sessions 

e Teachers have time incentive to explore data 


KPIs e Number of educator preparation programs that revise curriculum 
e Number of trainings offered in schools 
e Percentage of practitioners that attend trainings 


e Blocks of time to explore data 
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Dimension VI: Student Support 


The Student Support dimension measures six subdimensions including Student Orientation & Support, Equity, 
Accessibility, Compliance Standards, Social Emotional, and Innovation. 


It is clear in survey and interview findings that student support is a high priority. Pre-COVID, respondents indicated 
a diversity of on-site support services, and while post-COVID survey responses remain largely positive, 
discrepancies and changes, supported by interview data indicate that schools were not ready to provide such 
services remotely. The importance of doing so was clearly recognized, and many respondents indicated that they 
implemented immediate responses within their capabilities, and are planning for innovative solutions in the 
future. Implementation, evaluation and continuous improvement of these solutions is necessary to ensure 
optimal and equitable student support following the pandemic. 


Overall findings indicate that there is a high level of awareness among administrators, staff, and teachers 
regarding holistic student support needs. An interesting trend in this dimension was that for every question, 
‘agree’ responses were far more frequent than ‘strongly agree,’ while responses indicating disagreement or strong 
disagreement were approximately even. Both quantitative and qualitative data findings show that while there are 
clear needs for improvement in all subdimensions, robust on-site services were previously implemented and there 
are innovative ideas and processes in development to extend these support services to students in a digital 
environment. 


In the further development and implementation of appropriate and effective student support, the most evident 
gaps are between administrators, staff, and teachers and parents/students. Qualitative data indicate that this may 
be at least partially due to the rapid response to COVID, requiring focus on knowledge provision as the top priority. 
Additionally, there may be a need for increased assessment of needs and communication to parents and students 
regarding the availability of support. Some schools are already engaged in addressing these needs, and the 
recommendations for this dimension support the continuation of this existing work and provide potential 
solutions for further development and improvement. 


Subdimension: Student Orientation and Support 


The student orientation and support subdimension assessed responses to pre- and post-COVID situations 
regarding orientation to classes and technologies as well as academic and technological support, attendance to 
interpersonal issues, and legal and safe behavior related to technology. 


Administrators, staff, and teachers are largely in alignment with their perceptions of the provision of orientation, 
class design and activities, and technologies, both pre- and post-COVID. Students and parents also aligned in their 
agreement or disagreement, but at lower rates than administrators, staff, and teachers. There is a clear 
discrepancy in the perception of technological and academic orientation and support provided between school 
employees and parents/students. While improvements are indicated in most areas of this subdimension, post- 
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COVID perceptions exceed pre-COVID, with one notable exception. Pre-COVID, students agreed or strongly agreed 
that they were provided academic support at rates of 59.6%, and disagreed or strongly disagreed at 24.3%. Post- 
COVID, students agreed or strongly agreed at 39.9% and disagreed or strongly disagreed at 42.5% (Figure 55). 


STUDENTS ARE PROVIDED INFORMATION ON CLASS TECHNOLOGY 
AND ACADEMIC SUPPORT RESOURCES. 
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Figure 55 


These discrepancies are at least partially explained by interview data. Parents repeatedly indicated that 
orientation and technology support were not present either before or after COVID, which may be reflected in their 
survey responses. One administrator supported survey findings and said, regarding technology orientation and 
support, “Honestly, no. It was more like a shock and we focused on spreading knowledge to all students. In 
addition to that, they self-studied and improvised.” While schools are aware of this concern, there are some 
solutions presented by administrators during their interviews. One said “With the help of two computer science 
teachers, and the digital transformation coordinator, we made a team for the online platform trainings, we 
repaired and opened the two computers labs we have in the school, and then we hold training courses in 
technology for students who have some difficulties in using technology.” Another noted the importance of 
providing age-appropriate support, saying “I have a plan, which includes mainly training the elementary students 
on using technology in a way that suits their ages. That’s why I’ve bought some iPads, to train students on using 
them. During the pandemic we’ve focused on the middle school students and trained them on how to use 
technology using their smartphones.” Finally, interview responses also reflect an awareness and understanding of 
the need for academic support. One teacher said, “Personally, | prepared my students and | talked to them on how 
to use the online platforms and about the exams, | also have them demo tests to help them understand the way 
online exams will be, which made them more engaged.” 


Student and parent responses are consistent regarding interpersonal issues in online courses, as are parent 
perceptions. However, the need for improvement is clearly indicated. Both students and parents agreed or 
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strongly agreed that support is available for interpersonal issues at less than 39%, both pre- and post-COVID, and 
disagreed or strongly disagreed at similar rates of 39.6%-41% (Figure 56). 


SUPPORT IS AVAILABLE TO ADDRESS INTERPERSONAL ISSUES IN 
ONLINE COURSES. 
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Regarding legal, ethical, and safe behavior online, pre- and post-COVID responses were similar, but parents clearly 
indicated that improvements are needed, and this will be especially important if online education continues to be 
a primary method due to the pandemic. Just over half of parents agreed or strongly agreed that these practices 
are in place both prior to and after COVID, but nearly a quarter (22.4%) prior and and just over a quarter (25.4%) 
indicated that they disagreed or strongly disagreed. 


Recommendation: Provide mandatory orientation to online courses, including 
technology, class design, academic support resources, and technological support 
resources. 


Target Teachers, Students 

Audience 

Needs Clear and comprehensive orientation provided that includes all necessary elements of 
technology, class design, academic support resources, and technological support resources 

Initiatives Develop and implement comprehensive online course orientations 

Goals To increase student awareness of and ability to access support needs for courses as well as to 
effectively use learning technologies used in courses 

Actions Develop and implement comprehensive online course orientations 

CSFs Orientations are developed, include all necessary elements, and provided to students 

KPIs e Percentage of students that complete online course orientations 
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e Percentage of students that indicate awareness and use of support resources 
e Number of support tickets related to students’ ability to effectively navigate courses 
and use course technologies 


Recommendation: Ensure that academic support resources are available and may be 
accessed online. Examples include digital libraries and databases, tutoring services, etc. 


Target Administrators, Staff, Teachers, Students 
Audience 
Needs Provide academic support resources that are accessible online 
Initiatives e Develop sufficient academic support resources 
e Implement digital access for academic support resources 
Goals To increase student awareness of and ability to access academic support resources 
Actions e Develop academic support resources 
e Implement digital access for academic support resources 
CSFs Academic support resources are developed and accessible to students online 
KPIs e Percentage of students that indicate awareness of available academic support 
resources 


e Students access/engagement with academic support resources 


Subdimension: Equity 


In this subdimension, administrators, staff, and teachers all indicated that pre-COVID, approximately half or more 
respondents believed that there were “resources in place to ensure all students, including underrepresented 
students, have an equitable experience” (Figure 57). Post-COVID, agreement related to these perceptions 
increased slightly, and disagreement decreased. This is a critical area, particularly during COVID, when students 
are largely or wholly learning online with the potential for rapid change. Though there was consistency in 
response, both quantitative and qualitative responses indicate additional needs. 


Os Cc ONLINE LEARNING 
CONSORTIUM 


RESOURCES ARE IN PLACE TO ENSURE ALL STUDENTS, INCLUDING 
UNDERREPRESENTED STUDENTS, HAVE AN EQUITABLE 
EXPERIENCE. 
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Qualitative data provided substantial context surrounding both the challenges and existing efforts to support 
student equity. Efforts toward equity appear to be both high-priority for schools as well as holistic in scope. 
Several respondents indicated that they are better able to provide these services onsite, with one administrator 
saying “We, at our school, counted students with difficult economic situations who cannot afford to buy 
computers or use Internet. We bought 10 computers and put them in the library, resource room and computer lab. 
Then, we asked such students to perform all their duties in any possible time, whether during the break time or 
almost before leaving.” The challenge of providing support in the digital environment was articulated by several 
respondents. One teacher noted “We do not know whether their parents have provided them with what they need. 
We do not know whether or not their fathers have low income or whether or not they have internet at home. There 
are many obstacles in this regard.” Despite these challenges, some schools have implemented solutions to assess 
and address equity. One example from an administrator was “We send a confidential survey for students in which 
they fill their data financial-wise, health-wise, and else. And the counselor gathers the surveys and collects 
data...the school provides financial support to the student according to the school ability in order to buy her 
necessary needs.” In addition, respondents indicated support for ensuring that students are able to learn at home 
if there are cultural barriers. One representative statement from an administrator articulates these needs and 
solutions: “There are some students that do not enter [the course], so the counselor communicates with the 
student’s parents to convince him. There are some parents when they hear about the problems that happen, they 
ban their daughters from using the phones. If the parent is afraid, we inform him when the student will perform 
the homework and send a special mobile message to him so the student can do her homework.” 


In addition to technological equity, qualitative interviews indicate that schools are invested in providing holistic 
support for equity, including financial, health, and academic aspects of need. Responses on many of these items 
are discussed in detail in the social emotional subdimension, and academic equity is particularly relevant to this 
subdimension. “I think for the digital equity, | think there are a number of solutions that some of them are already 
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implemented and some of them may be under development. But one of the solutions that was proposed is to 
design learning journeys that suitable for different, like those with special needs, those with high IQ. Well, they can 
do more. So, you need to give them something suitable to their capabilities and abilities...| think the whole idea is 
that to design different learning paths and learning journeys to students with different capabilities.” 
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Recommendation: Develop and implement measures to evaluate student support needs 
related to equity, including financial, health, and academic areas. Ensuring 
confidentiality of responses is critical. Note that there is some overlap in this dimension 
between the equity and social emotional subdimensions and further recommendations 
are provided specific to social emotional supports in that section. 


Target Administrators, Staff, Teachers, Parents, Students 

Audience 

Needs Develop and implement evaluations to assess student support needs related to the provision of 
equitable education opportunities, including financial, health, and academic areas 

Initiatives Develop and implement appropriate evaluations to assess student support needs 

Goals To increase the ability of schools to identify student support needs and provide solutions 

Actions Develop and implement appropriate evaluations to assess student support needs 

CSFs e Development of student support evaluations 


e Datacollection from student support evaluations 
KPIs e Evaluations completed 
e Data collected 


Recommendation: Based on evaluations of student support needs, develop solutions to 
provide student support that may be accessed by students either digitally, or that 
account for the inability for students to be on-site. 


Target Administrators, Staff, Teachers 

Audience 

Needs Develop and implement student support solutions that are not reliant on on-site delivery 
methods 

Initiatives Develop and implement student support solutions that may be accessed at a distance 

Goals To increase the ability of schools to provide student support regardless of whether students 
are onsite or online 

Actions Develop and implement appropriate student support and delivery methods 

CSFs e Development of student support services that are accessible via distance 


e Provision of student support services that are not reliant on on-site presence 
KPIs e Implemented student support services 
e Number of times student support services are accessed by students 
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Subdimension: Accessibility 


Responses to the accessibility subdimension follow those previous, and indicate that there is room for 
improvement related to the policies, processes, and resources available to support students with disabilities. At 
rates of agreement or strong agreement of less than 63%, and disagreement or strong disagreement at 13.5- 
22.5%, and particularly in the post-COVID environment, attention to accessibility is recommended (Figure 58). 


POLICY, PROCESSES, AND RESOURCES ARE IN PLACE TO SUPPORT 
STUDENTS WITH DISABILITIES. 
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Figure 58 


Interview data indicate support for accessibility needs onsite. For example, one administrator said “We make sure 


that classes are suitable for all of the students...some students have sight problems which led us in some of the 
times to change the lighting in the classroom. We care about our students, that’s why we support students who 
face academic issues from teachers and supervisors.” It may be that the necessary focus prior to COVID was 
primarily on addressing on site needs, and interviews do not reflect digital accessibility, lending strength to the 
assessment that this area should be a focus. 
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Recommendation: Ensure that online courses follow accessibility standards. 


Target Administrators, Staff, Teachers 

Audience 

Needs Ensure that online courses follow accessibility standards to support students with disabilities 

Initiatives Review the use of accessibility standards in online courses and ensure that gaps are 
addressed 

Goals To increase the ability of students with disabilities to fully access and participate in online 
learning 

Actions Review the use of accessibility standards in online courses and ensure that they are duly 
met/maintained to the fullest possible extent 

CSFs e Courses are reviewed for adherence to accessibility standards 


e Any gaps identified are addressed 

KPIs e Percentage of courses reviewed for adherence to accessibility standards 
e Percentage of gaps identified and addressed 
e Access and participation by students with disabilities in online learning 
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Subdimension: Compliance Standards 


As might be expected, responses to the compliance standards subdimension are closely aligned to the 
accessibility subdimension. Findings indicate that there is room for improvement related to compliance with 
accessibility standards. Prior to COVID, 50-60% of administrators, staff, and teachers indicated that they agreed or 
strongly agreed that “online classes demonstrate compliance with accessibility standards,” while 14-18% 
disagreed or strongly disagreed. These ranges remained consistent, though there was improvement across all 
groups’ perception post-COVID, including an increase among staff in agreeing or disagreeing at 65.5% (Figure 59). 


ONLINE CLASSES DEMONSTRATE COMPLIANCE WITH 
ACCESSIBILITY STANDARDS. 
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Figure 59 


Recommendation: Ensure that accessibility standards (as described the accessibility 
subdimension) are in alignment with compliance standards. 


Target Administrators, Staff, Teachers 

Audience 

Needs Ensure that online course accessibility is in compliance with standards 

Initiatives Review online courses for compliance with accessibility standards 

Goals To increase the compliance of online courses with accessibility standards 

Actions Review courses for compliance with accessibility standards and address areas where 


compliance is not met 
CSFs e Courses are reviewed for compliance with accessibility standards 
e Areas in which compliance are not met are addressed 
KPIs e Percentage of courses reviewed as compliant with accessibility standards 
e Percentage of gaps that are identified and addressed 
e Percentage of compliance standards met in online courses 
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Subdimension: Social Emotional 


Administrators, staff, and teachers consistently agreed or strongly agreed, between 58.3-64.8% that “students are 
provided with opportunities for social-emotional learning and support, such as building relationships, culture, 
and community in a remote setting.” Disagreement or strong disagreement was indicated at 16.3%-19.5%. These 
responses were stable both pre- and post-COVID, though post-COVID responses indicated slight improvement. 
Student responses prior to COVID also followed this pattern, with 60.1% agreeing or strongly agreeing that 
“support for my well-being (including counseling, etc.) was available to me,” and 23.3% disagreeing or strongly 
disagreeing. The most notable finding was the significant drop in student perceptions of this item post-COVID, 
with agreement or strong agreement lowering by 20 percentage points, to 39.6% and disagreement or strong 
disagreement increasing to 43% (Figure 60). 


SUPPORT FOR MY WELL-BEING WAS AVAILABLE TO ME. 
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Figure 60 


Additionally, students were queried about “opportunities to build relationships and community online” and 
whether they were able to make friends with their classmates and others online. Responses both pre- and post- 
COVID were consistent, though they indicate the need for more opportunities and support in these areas. 
Agreement or strong agreement ranged from 37.6-46.9%, and with approximately equal disagreement or strong 
disagreement at 36.8-43.1%. 


Though responses specifically relevant to the survey items noted above indicate the need for improvement, 
particularly post-COVID, considerations of student support are holistic, and include social emotional components, 
including physical and mental health as well as financial situations. Many respondents noted that their schools 
provide access to medical services, and several noted that their schools educate students on health and hygiene, 
and provide supplies as needed. Additionally, respondents repeatedly indicated that they monitor social 
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emotional support needs to identify students who require services. One administrator said “We communicate 
with our students regularly, the counselor is responsible of following up with students, and keeping in touch with 
the teachers in case any issue or changes occurred on any of the students. We also keep communicating with the 
parents.” Direct financial support is indicated, as well as support for basic needs such as meals and clothing. 
Interview participants identified that providing services has been difficult during COVID. One administrator said “It 
is hard to make sure that every student is able to have support. You can make sure that the students have the 
important things while they are at school. But you do not have the right to access students’ rights. The conditions 
of families are different. But in general, schools they give hand the student to make sure to get everything they 
need, even in all life aspects, and everyone tries their best and gives everything they can for the students. But 
when it comes to online learning, when the student are at home, it’s really hard to make sure that everyone has 
the same support services because you do not have the right to access their lives to help them.” Another said “In 
e-learning, none of them [support services] were provided. Each student had to deal with his own personal 
resources and abilities.” However, awareness of support needs are present, and while it may not have been 
possible to provide them during COVID, interviews indicate that schools are actively developing situational 
assessments and focusing attention on solutions for providing social emotional support. 


Recommendation: Develop and implement measures to evaluate student support needs 
related to social emotional areas. Ensuring confidentiality of responses is critical. Note 
that there is some overlap in this dimension between the equity and social emotional 
subdimensions and further recommendations are provided specific to equity supports in 
that section. 


Target Administrators, Staff, Teachers, Parents, Students 

Audience 

Needs Develop and implement evaluations to assess student support needs related to the provision 
of social emotional services 

Initiatives Develop and implement appropriate evaluations to assess social emotional student support 
needs 

Goals To increase the ability of schools to identify social emotional student support needs and 
provide solutions 

Actions Develop and implement appropriate evaluations to assess social emotional student support 
needs 

CSFs e Development of social emotional student support evaluations 


e Data collection from student support evaluations 
KPIs e Evaluations completed 
e Datais collected 


Oia C ONLINE LEARNING 
CONSORTIUM 


Recommendation: Based on evaluations of social emotional student support needs, 
develop solutions to provide support that may be accessed by students either digitally, or 
that account for the inability for students to be on-site. 


Target Administrators, Staff, Teachers 

Audience 

Needs Develop and implement social emotional student support solutions that are not reliant on on- 
site delivery methods 

Initiatives Develop and implement social emotional student support solutions that may be accessed at a 
distance 

Goals To increase the ability of schools to provide social emotional support to students regardless 
of whether students are onsite or online 

Actions Develop and implement appropriate social emotional student support and delivery methods 

CSFs e Development of social emotional student support services that are accessible via 

distance 


e Provision of social emotional student support services that are not reliant on on-site 
presence 
KPIs e Implemented student support services related to social emotional needs 
e Number of times student support services related to social emotional needs are 
accessed by students 


Recommendation: Provide course-based opportunities for the social emotional support of 
students, such as opportunities to build relationships and community online, in addition 
to intervention strategies as noted above. 


Target Teachers, Students 

Audience 

Needs Opportunities to develop social emotional support in normal course engagement, including 
relationship- and community-building 

Initiatives Develop and implement opportunities for students to build relationships and communities 
within their courses 

Goals To increase social emotional support of students through opportunities for relationship- and 
community-building 

Actions Develop and provide opportunities for students to build relationships and communities in the 
context of their regular course engagement 

CSFs Opportunities are present for students to build relationships and communities within their 
online courses 

KPIs e Students’ social emotional support opportunities within their online courses 

e Student participation in such opportunities to effectively build social emotional 
support systems 


Subdimension: Innovation 


Innovation related to student support focused on communication modes and tools. Administrators, staff, and 
teachers were consistent in their responses both pre- and post-COVID, though there were slight improvements 
after COVID. Respondents agreed or strongly agreed at rates of 61.7-69.9%, and disagreed or strongly disagreed at 
11.4-16.8%. 


Recommendation: Include innovative communication modes and tools for supporting 
students in online environments. 


Target Administrators, Staff, Teachers 

Audience 

Needs Increased and effective communication in online courses 

Initiatives Discover and implement innovative methods of communicating with students in online 
courses 

Goals To improve communication methods in online courses 

Actions Discover and implement innovative methods of communicating with students in online 
courses 

CSFs Innovative communication methods are implemented in online courses 

KPIs Students recognize the presence and effectiveness of communication in online courses 


Dimension VII: Training & Support 


The Training and Support dimension measures five subdimensions including Technical Assistance, Professional 
Development, Orientation (Student & Parent), Mentoring, and Innovation. 


Training and support readiness appears to have been commensurate with the pace of online learning 
development and delivery in a pre-COVID environment. Post-COVID, survey and interview results indicate that 
although training and support related to online learning was not fully prepared for the shift due to the pandemic, 
sufficient resources existed for prior needs and actions were taken to provide them for the COVID and post-COVID 
environments. Focus in all areas, building upon training and support initiatives continued or launched during 
COVID is recommended for improved readiness for effective online learning following the pandemic. 


Overall findings within the Training and Support dimension indicate that school administrators, staff and teachers 
are satisfied with the relative activities and opportunities provided. By and large, school administrators, staff and 
teachers agreed or strongly agreed, at around 50%, that the queried practices in each subdimension are 
introduced and effectively actualized. The most impactful finding in measuring the Training and Support 
dimension is represented by the rise in satisfaction and strong satisfaction indices from pre- to post-pandemic 
phases, and the concurrent decline in the general and strong disagreement rates, alongside the proportion of 
neutral attitudes. 
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The majority of school administrators, staff and teachers agreed that teaching online courses has sufficient 
training and support for the transition to distance learning. These activities include continuing practice-oriented 
ministerial and non-ministerial courses on training, assistance, and support to prepare for classes and teaching 
online. There are some small discrepancies between perceptions of administrators, staff and teachers to the 
degree of their agreement, but they generally trend similarly positively. 


While students were not asked to elaborate on the items from the included subdimensions, since it would be 
unlikely that they would have insight into these areas, interview questions reveal concerns that teachers were not 
always adequately prepared nor had sufficient technical assistance to support students and have high quality 
subject course delivery. 


Not so long ago, teachers could have voluntarily taught online relying mainly on their personal interest in 
technology and willingness to develop online teaching skills. Today, transforming into an online teacher is a 
ubiquitous requirement, since education systems worldwide shift their focus to blended learning. Moreover, the 
overwhelming majority of students are denizens of the digital environment, and it is crucial that teachers keep up 
with the times and emergencies. Thus, the teacher training and supporting system must prepare educators for the 
new reality. 


Subdimension: Technical Assistance 


Measures of satisfaction with technical assistance both pre- and post-COVID among school administrators, staff 
and teachers display an obvious positive trend toward the upside in general and strong agreement rates, with the 
majority of participants indicating that they are overall satisfied or very satisfied about the survey items. 
Conversely, the degree of disagreement reduced through the pandemic, which is well supported by quantitative 
data thereto (decline of approximately 2%, corresponding to over 1000 respondents to the survey) (Figure 61). 
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TECHNICAL ASSISTANCE IS PROVIDED FOR TEACHERS DURING 
ONLINE CLASS DEVELOPMENT AND ONLINE TEACHING. 
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While students did not respond to this subdimension, the fact that teachers faced technical issues, concerning 
recorded lessons, 3D movie making, online scientific experimenting, education platforms (Future Gate or 


alternative) and smartboard utilization, remote exercising and evaluation, evidentiated the fact that, in principle, 


everyone needed more sufficient technical assistance to use advanced technologies and continuingly emerging 


instrumentality for teaching, learning and interaction. 


Recommendation: Develop technology-facilitated web-mediated guidance and 


instruction program to integrate principles for meaningful and sustained online learning 


in addition to addressing workplace barriers to sustained learning through non-stop 
training. Feedback from all members of the education process is a pre-condition for 
ensuring appropriate technical assistance. 


Target Administrators, Staff, Teachers, Students 
Audience 
Needs Further improve technical assistance mechanisms for educators and students during online 
activity development and online teaching and learning 
Initiatives e Create a centralized support space through a website/education portal 
e Resources explicitly shared and stressed by teachers in syllabi and online course, and 
school 
Goals e Toremove technical obstacles for teachers’ pedagogical success and students’ 


learning progress 
e Todevelop and implement online resources for common use and contribution 
Actions e Create expert teams of stakeholders at schools including student members to inspire 
and guide the development of most required resources 
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e Create a centralized portal/website with curated resources (e.g., guides, videos, online 
video lectures, frequently asked questions) and support mechanisms (e.g.: live chats, 
chatbots, fora) 

e Communicate resources through Future Gate, online courses, and syllabi 

CSFs e Stakeholder expertise, adequate response to student needs and teacher requirements 

e Centralized website available with curated insightful resources and support 
mechanisms, and updated to ensure actuality of materials and urgency of assistance 
provided 

e Resources shared through Future Gate or alternative education platforms, online 
courses, and syllabi 


KPIs e Page views and hits on website 
e Future student technical issues duly collected and considered to update the resource 
base 


e Common support language in syllabi and online courses 
e Shared pedagogical elaborations and successful resources 
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Subdimension: Professional Development 


Measures of satisfaction with professional development both pre- and post-COVID among school administrators, 
staff and teachers validated the growing satisfaction with the professional development opportunities ensured, 
with the majority of respondents satisfied (51%) or very satisfied (14.5%). It is noteworthy that the proportion of 
those strongly agreeing on the relative survey items rose by 3% from pre- to post-COVID times. 


Overall, the majority of school administrators, staff, and teachers express that those teaching online receive 
training assistance and support to prepare for course development and remote education activities. 
Administrators and staff report that they agree or strongly agree at the rates of 54.6% and 15.5%, respectively, 
that professional development for online teaching and learning was continuingly and substantially ensured 
(Figures 62, 63). 


TEACHERS RECEIVE TRAINING, ASSISTANCE, AND SUPPORT TO 
PREPARE FOR CLASSES AND TEACHING ONLINE. 
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TEACHERS ARE PROVIDED ONGOING PROFESSIONAL 
DEVELOPMENT RELATED TO ONLINE TEACHING AND LEARNING. 
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Expert Recommendation: Online learning will further increase in the future, and school 


administration and staff need to identify ways to provide ongoing support to their 
pedagogical staff in order to prepare them to teach in online learning environments. This 
includes subject-related content development, ability to interact with students 
efficiently, capacity to utilize new technology, etc. Thence, ensuring ongoing professional 
development, training and assistance in pedagogy and technology is essential for schools 
to develop the knowledge, skills, and abilities to design and teach online classes. This will 
gradually result in solid and widespread improvement within and beyond the school level 


itself. 
Target Audience Administrators, Staff 
Needs Regular access to comprehensive and up-to-date professional development 
Initiatives Continuing professional subject- and technology-related training and support for teachers 
to develop the knowledge, skills, and abilities essential for designing and giving online 
classes 
Goals To organize non-stop collaborative and job-embedded professional development to 
increase efficiency and confidence of teachers 
Actions e Conduct needs assessments using data from staff surveys to identify areas of 
professional development most requested and anticipated by teaching staff 
e Identify mentors and coaches (either in-house or through expert providers) to 
guide and support teacher learning specific to online education 
e Ensure access to demo online lessons, assignment co-designing, co-planning, 
curricular resources, including lesson units and assessment tools 
CSFs Sufficiently and adequately prepared educators to teach online through accessible and 
adequate collaborative and job-embedded professional development 
KPIs Teacher efficiency and self-efficacy 
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Expert Recommendation: Time can be an advantage and a challenge in an online learning 
environment. A recommendation might be moving a long synchronous training into a 
modular, asynchronous format. This will ensure opportunities for teachers to complete 
training activities and assignments with thorough approach and attention, when they 
have available time for that. Flexibility is a virtue of remote learning, thus, teachers 
should, as well, benefit from the “new reality” in their personal and professional 
advancement. Positive is the fact that overall dissatisfaction of school administrators, 
staff and teachers about adequacy of time to engage in professional development, 
decreased through the pandemic from 18.9% pre-COVID to 15.6% post-COVID. 


Target Audience Administrators, Staff 


Needs Increase opportunity for teachers to complete professional development and remove 
barriers to supporting online resources and instructions 

Initiatives Extend and reinforce teachers’ professional advancement and promote educators’ capacity 
building 

Goals To provide teachers with adequate time to learn, practice, implement, and reflect upon new 


strategies that facilitate changes in their professional practice 


Actions e Evaluate the use of time and redesign school schedules to increase opportunities 
for teachers’ professional development, self-education and co-working 

e Organize active learning, aligned with national policies and school curricula, and 
provide sufficient learning time for teachers to engage and progress 

e Ensure availability of meaningful training sessions of convenient structure and 
duration within an academic year 

e Break longer training sessions into more manageable elements that can be 
completed over a longer period of time 


CSFs e Modular content that can be completed in less time 

e Technical support structure for short courses 

e Expert group of mentoring teachers and staff to ensure feedback on curated 
resources 

e Teachers made aware of resources through a variety of channels (e.g., school 
website, Future Gate or alternative online platforms, emails, etc.) 


KPIs e Time teachers have to learn, practice, implement, and reflect upon new teaching 
strategies 

e Number of opportunities to engage in learning around a single set of concepts or 
activities within an academic year 

e Students learning outcomes as a result of efficient teacher professional 


development schemes 
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Subdimension: Orientation 


Measures of satisfaction with orientation both pre- and post-COVID among school administrators, staff and 
teachers show positive movement, with the majority of participants indicating that they are satisfied or very 
satisfied and their numbers and shares increasing through the pandemic. 


Very close to or even over half of administrators, staff, and teachers felt that educators received a sufficient 
technology and instructional design orientation. Overall, administrators and staff reported satisfaction before and 
after the pandemic, at 65% and 68.4% respectively. Teachers’ aggregated satisfaction rates pre- and post-COVID 
featured 59.8% and 62.9%, accordingly. At the same time, teachers' share of those very satisfied is recorded less 
than those of administrators and staff with higher “strongly agree” figures and rates (Figure 64). 


TEACHERS ARE PROVIDED ORIENTATION/TRAINING ON 
TECHNOLOGY AND INSTRUCTIONAL DESIGN USED IN THE 
DEVELOPMENT OF ONLINE CLASSES. 
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Aschool administrator from Jeddah shared specific encouraging experience, stating that the school educators 
start an academic year 2 weeks before students, when they have 2 courses per day. Several training sessions are 
held, particularly on the essentials of using a smartboard. All staff members attend training with ICT instructors in 
the labs. At the beginning of each academic year, the school teachers undergo training on Future Gate. Intensive 
training sessions are additionally available for new teachers. Administration starts the school year before 
teachers, so the first have one week, which includes much training on technology use. 


Sear 


Expert Recommendation: Most teachers state that they received abundant, 
comprehensive and sufficient pedagogical and technology-specific orientation. Quality 
interviews provide considerable evidence of the numbers obtained. Throughout the 
Kingdom of Saudi Arabia, technological equipment is quite similar across schools and 
regions. As mentioned by several respondents in the quality interviews, schools in the 
Kingdom do not lack technologies or instrumentality, what is required is relevant and 
valid expert training and support. 


Target Audience — 


Ministry of Education, Education Departments, NELC 


Needs 


"Improve access to orientations for widespread and promising technologies 


Initiatives 


Develop, test and share common orientation modules for schools 


Goals 


To decrease resistance and challenges among teachers for online learning technologies and 


improve student outcomes 


Actions 


Generate shared vision on high-quality teaching standards, instruction and 
evaluation 

Identify widespread technologies and determine most critical for development 
Partner with relevant educational institutions to develop and pilot orientation 
modules 

Use best national and international practices of teacher professional development, 
which have raised student performance, for better understanding of effective and 
rational orientation and support 


CSFs 


Education-critical technologies identified 

Regularly updated orientation modules to include new technologies 
Piloted at schools to provide reasonable and actual feedback 

Orientations easily accessible and shared with key personnel at institutions 


Number of modules developed and updated periodically 
Number of modules accessed and implemented by institutions 
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Subdimension: Mentoring 


Measures of satisfaction with mentoring in the pre- and post-COVID times among school administrators, staff and 
teachers designated increase in overall satisfaction across the involved target groups, while negation rates 
decreased through the pandemic (total strong disagreement percentage decline by 3% from pre- to post-COVID). 
The majority of participants stated that they were satisfied (total rate of 49.1%), however the second popular 
response to the survey was neutral, though featuring a decrease by over 2%. It is important and encouraging to 
note that post-COVID, the “strongly agree” evaluation of the mentoring opportunities showed the highest rise 
from the pre-pandemic period (Figure 65). 


PEER MENTORING RESOURCES AVAILABLE TO ONLINE CLASS 
TEACHERS. 
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Recommendation: Not all teachers receive sufficient peer mentoring resources, some 
have never had the experience. Peer mentoring provides opportunities to support each 
other through the shift to remote and fully-online teaching and learning as well as basic 
life needs and continuing to grow as an academic. A centrally-coordinated virtual 
mentoring program can ensure opportunities for more consistent and viable connection 
and support. 


Target Administrators, Staff, Teachers 

Audience 

Needs Develop, sustain and improve opportunities for peer mentoring 

Initiatives Design a virtual peer support (peer observation/peer evaluation) and mentoring program for 


collegial learning groups 
Goals To provide flexible, continuing and consistent mentoring, coaching and sustained 
engagement in collaboration 
Actions e Communicate with teachers to identify acute mentoring needs 
e Connect mentors with mentees based on their needs and preferences 
e Provide resources and incentives where possible and appropriate 
CSFs e Teacher needs and preferences identified 
e Mentors and coaches approved and assigned 
e Schemes < procedures and criteria duly communicated to teachers-mentees 
e Feedback collected and considered to improve program where possible 
KPIs e Number of mentors and mentees 
e Academic output (e.g., publications, grants, etc.) 
e Staff retention 
e Teachers’ health and wellbeing 
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Subdimension: Innovation 


Measures of satisfaction with training and support innovation both pre- and post-COVID among school 
administrators, staff and teachers followed the positive trend toward the upside in general and strong agreement 
rates, with the majority of participants (total of 51.3%) indicating that they are overall satisfied with the provided 
opportunities to select new tools and pilot emerging technologies. It is noteworthy and remarkable that strong 
agreement on the survey innovation item is characterized by the highest rise in rates (upwards by over 3% post- 
COVID). Quite the contrary, dissatisfaction indicators decreased, as well as the proportion of neutral responses. 


The majority of administrators, staff and teachers reported that educators were duly informed and encouraged 
about advanced technology and emerging tools. Post-COVID, administrators and staff reported at 52.8% and 
teachers similarly reported at 79.9% in aggregate and teachers as a sole group evidentiated their satisfaction at 
50.4% (post-COVID) (Figure 66). 


TEACHERS ARE PROVIDED OPPORTUNITIES TO SELECT NEW TOOLS 
AND PILOT EMERGING TECHNOLOGIES. 
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Expert Recommendation: Despite the expanding access to advanced technology and the 
fact that tech savvy teachers join education systems and communities, teachers still 
prevalently report “moderate” not “frequent” or “widespread” use of technology as 
instructional instrumentality (due to policies, age, infrastructure issues, etc). It is 
quintessential that, if technology is used as a teaching and learning tool, correlating to 
educational goals and assessment, embedded within a comprehensive pedagogical 
approach and guided by efficient techniques, technology can ensure better instruction, 
student collaboration, motivation and learning outcomes. The great way to support 
awareness and use of new technologies is through a teacher learning community 
program. Such programs can be led by external experts or members of the school 
teaching staff, or can even be coordinated centrally through regional education 
department(s) and teaching and learning centers. 


Target Audience Administrators, Staff, Teachers _ 


_ Needs _ Greater awareness and more impactful integration of advanced technologies 
Initiatives _ Develop a teacher learning community program 
Goals ‘To build capacity around online learning tools and innovative pedagogical approaches 
Actions | e Teachers form a community (or multiple communities) 


e Educational institutions recruit teachers/pedagogues into communities and 
provide centralized coordination 


CSFs e Community groups formulate their common and accepted objectives, set regular 
meetings, share resources, best practices and ideas 

e Clear and well-communicated expectations for program, regular meetings, a focal 
point to coordinate the group and ensure all needs and requirements are met, 
incentives for participation (resources, awards, grants, recognition, etc.), 
recruitment from all departments to grow best performers 


KPIs e Number of groups 
e Participation in groups 


e Dissemination of knowledge and skills 


Dimension VIII: Evaluation & 
Continuous Improvement 


The Evaluation and Continuous Improvement dimension measures five subdimensions including Evaluation of 
Course Outcomes and Program Quality, Student Satisfaction, Teacher Satisfaction, Staff Satisfaction, and 
Innovation. 
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Respondents indicated that improvements to readiness are needed regarding evaluation and continuous 
improvement. Areas requiring the most attention are evaluation of course outcomes and program quality, 
including the development and implementation of clear processes and metrics, as well as review and process 
implementation of student satisfaction measures. Both teacher and staff satisfaction increased following COVID, 
though recommendations are provided to further improve the ability to collect and constructively use feedback. 


Survey questions focused on evaluation and continuous improvement were included for administrators/staff, 
teachers, parents and students. Qualitative data from interviews was also obtained from administrators/staff and 
teachers. 


Subdimension: Evaluation of Course Outcomes & 
Program Quality 


Regular review of online classes and programs are necessary to ensure that content is current and meeting the 
needs of the learners. A review of survey responses from administrators, staff and teachers reflects agreement that 
in most cases, that the procedures and policies for evaluating and improving online courses and programs were in 
place before and after COVID-19 (see Figure 67). Given the percentages that were neutral or not in agreement, this 
may still be an opportunity for improvement. As seen in Figure 68, similar results were identified regarding the 
question “intended learning outcomes at the class and program level are reviewed regularly to ensure alignment, 
clarity, utility, appropriateness, and effectiveness.” 


THE INSTITUTION HAS DEVELOPED A PROCEDURE AND/OR POLICY 
FOR EVALUATING AND IMPROVING ONLINE CLASSES AND 
PROGRAMS ON A TIMELY BASIS. 


mAdministrators mStaff mTeachers 
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Figure 67 
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INTENDED LEARNING OUTCOMES AT THE CLASS AND PROGRAM 
LEVEL ARE REVIEWED REGULARLY TO ENSURE ALIGNMENT, 
CLARITY, UTILITY, APPROPRIATENESS, AND EFFECTIVENESS. 


mAdministrators mStaff mTeachers 
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Figure 68 


It appears that most administrators were satisfied that their programs used an evaluation process that applied 
specific established standards. The percent in agreement went up after COVID-19 to 63.6% agreement from 56.4% 
before COVID-19. Similarly, the number that were not in agreement about having a process in place went down 
after COVID-19 to 12.3% from 17.8% prior to COVID-19 (Figure 69). 
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ONLINE PROGRAMS ARE ASSESSED THROUGH AN EVALUATION 
PROCESS THAT APPLIES SPECIFIC ESTABLISHED STANDARDS. 
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Figure 69 


There is a lot of data available to help administrators make informed decisions especially in continuous 
improvement efforts. The majority of administrators indicated that data was being used to evaluate program 
effectiveness. The responses did show some improvement between pre-COVID (56.9%) and post-COVID (63.9%) 
efforts. However, there appears to be a disconnect between the survey results and responses by administrators 
and teachers in interviews that were conducted (Figure 70). 
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A VARIETY OF DATA ARE USED TO REGULARLY AND FREQUENTLY 
EVALUATE PROGRAM EFFECTIVENESS AND TO GUIDE CONTINUOUS 
IMPROVEMENT. 
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Figure 70 


During interviews with administrators and teachers, a number of questions were included regarding the use of 
evaluation methods for online learning programs before, during and after COVID-19. The interviews reflected that 
a variety of methods are used to determine course quality, although in some cases it was noted that no data is 
used in the program evaluation process, either before or during transition to remote learning. There are external 
resources (e.g., achievement exams, supervisor reviews, etc.) and internal sources of data (e.g., student and 
parent satisfaction, pre-exams, student grades, etc.) that were identified as regular sources of data. 


However, the measures used to determine quality in a course were not consistent or formal processes. As one 
administrator stated, “Through the teacher herself, | don’t interfere in courses and curriculums, but the teacher 
herself knows how to determine the quality of the course.” Another administrator noted, “Unfortunately, 
feedback is the weakest point in our schools. There’s no monitoring for the content presented to be developed.” 
This sentiment was echoed by some teachers, “Our school does not have a quality system or program. They have 
not set any quality programs.” Asked a similar question, another teacher stated, “The school doesn’t do anything, 
honestly.” While there similar responses noted, not all teachers indicate a lack of methods to review course 
quality. One teacher shared that the process at their school included a “...committee at school, consisting of the 
principal, the deputy and a group of teachers. They discuss each subject’s curricula, mistakes and books’ editing.” 


When considering what might change post-COVID, there was an interest in having clear procedures provided by 


the Ministry of Education. There were also specific ideas shared regarding what could be used to improve course 
quality in the future. As one administrator noted, “To check if the teacher has delivered quality online learning, 
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through observing, surveys for parents and checking the results, surveys and feedback from the students, and 
meetings.” 


Also important in an evaluation of an online program is a process to assess teaching performance. When surveyed 
regarding the processes in place to accomplish this, the majority (55.5%) were in agreement that this was 
occurring before COVID-19. After COVID-19, it appears that steps may have been taken to improve this process 
since the level of agreement increased to 64.6% (Figure 71). 


A PROCESS IS IN PLACE AND FOLLOWED FOR THE ASSESSMENT OF 
TEACHING PERFORMANCE IN ONLINE CLASSES. 
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Figure 71 


Administrators and staff indicated agreement that for the most part that there is a process in place that is followed 
for the assessment of support services for teachers and students. Similar to other survey responses, this indicated 
an increase in agreement after COVID-19 (65.3% Administrators; 69.6% Staff) as compared to prior to COVID-19 
(58.1% Administrators; 67.7% Staff). This reflects the likely steps that many institutions took during the transition 
to remote learning to prepare for upcoming school years (Figure 72). 
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A PROCESS IS IN PLACE AND FOLLOWED FOR THE ASSESSMENT OF 
SUPPORT SERVICES FOR TEACHERS AND STUDENTS. 
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Figure 72 


Recommendation: Provide processes, procedures, and policies on course and program 
evaluation, including regular review and updating. 


Target Teachers, Parents, Students 
Audience ne Oe 
Needs Develop and implement clear processes, procedures, and policies on course and program 


evaluation, including regular review and updating 


Initiatives Processes, procedures, and policies on course and program evaluation, including review and 
_ updating, are developed and implemented 


Goals _ To maintain and/or increase a high level of adherence to processes, procedures, and policies 
for course and program evaluation, and to continually improve these items through regular 
review and updating 


Actions e Develop and implement course and program evaluation processes, procedures and 
policies 
e Regularly review and update course and program evaluation processes, procedures, 
and policies 
CSFs e Course and program evaluation processes, procedures, and policies are in place 


e Course and program evaluation processes, procedures, and policies are regularly 
reviewed and updated as appropriate 


KPIs e Percentage of course and programs in alignment with established processes, 
procedures, and policies 
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e Frequency of review and updates for course and program processes, procedures, and 
policies 


Subdimension: Student Satisfaction 


Parents were given the opportunity to respond to several questions on the survey related to the evaluation of 
course outcomes and program quality both before and after the COVID-19 transition. When parents were asked if 
“parents and students were provided an opportunity to give feedback on their experiences with online classes,” 
there was little change noted in their responses pre- and post-COVID. Prior to COVID-19, parents indicated only a 
34.7% agreement (6.8% strongly agreed and 27.9% agreed) and 45.0% disagreement (21.9% strongly disagreed 
and 23.0% disagreed) with that statement. After COVID-19, that changed to 37.2% agreement (7.9% strongly 
agreed and 29.3% agreed) and 43.1% disagreement (22.0% strongly disagreed and 21.1% disagreed) (Figure 73). 


PARENTS AND STUDENTS WERE PROVIDED AN OPPORTUNITY TO 
GIVE FEEDBACK ON THEIR EXPERIENCE WITH THE ONLINE 
CLASSES. 
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Figure 73 


There was also consistency prior to and after COVID-19 in parents’ responses to the question “I was informed 
about how my feedback (and my students’ feedback) was used to develop or modify the online learning 
experience.” Again, most disagreed with this statement at both points and prior to COVID-19 only 33.7% 
agreement (6.8% strongly agreed and 27.0% agreed) and 46.4% disagreement (22.3% strongly disagreed and 
24.1% disagreed). After COVID-19, that changed to 36.2% agreement (7.8% strongly agreed and 28.4% agreed) and 
44.0% disagreement (21.9% strongly disagreed and 22.1% disagreed) (Figure 74). 
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| WAS INFORMED ABOUT HOW MY AND MY STUDENTS' FEEDBACK 
WAS USED TO DEVELOP OR MODIFY THE ONLINE LEARNING 
EXPERIENCE 
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Figure 74 


The last question that parents were asked related to student satisfaction was “class evaluations were conducted 
to collect feedback on the effectiveness of instruction and quality of online class materials.” Similar to other 
responses in this subdimension, parents remained fairly consistent prior to and after COVID-19. Most (43.8%) did 
not agree with this statement pre-COVID-19 with 22.3% indicating that they disagreed and 21.5% strongly 
disagreed. Only 34.8% were in agreement (6.8% strongly agreed and 28.0% agreed). After COVID-19, there was a 
slight increase (37.5%) in agreement with 7.9% indicating strongly agreed and 29.6% agreed. Disagreement with 
the statement decreased to 41.6% (21.2% strongly disagreed and 20.4% disagreed) (Figure 75). 
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CLASS EVALUATIONS WERE CONDUCTED TO COLLECT FEEDBACK 
ON THE EFFECTIVENESS OF INSTRUCTION AND QUALITY OF ONLINE 
CLASS MATERIALS. 


m@ Parents 
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Figure 75 


When asked to respond to the statement “I was given an opportunity to tell my teachers what | liked and didn’t 


like about my online courses,” student responses were more favorable prior to COVID-19. In fact, there was 51.9% 
agreement (15.9% strongly agreed and 36.0% agreed) prior to COVID-19 and 31.5% disagreement (14.3% strongly 
disagreed and 17.2% disagreed). After COVID-19, satisfaction seemed to drop with only 37.5% agreement (10.5% 


strongly agreed and 27.0% agreed) and 46.4% disagreement (23.6% strongly disagreed and 22.8% disagreed) 
(Figure 76). 


| WAS GIVEN AN OPPORTUNITY TO TELL MY TEACHERS WHAT | 
LIKED AND DID NOT LIKE ABOUT MY ONLINE CLASSES. 
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Figure 76 
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When administrators and teachers were asked about opportunities to assess student satisfaction with online 
classes and programs, the majority (55.7% Administrators and 54.5% Teachers prior to COVID-19; 63.6% 
Administrators and 50.3% Teachers after COVID-19) indicated that there were processes in place to gather this 
information (Figure 77). 


A PROCESS IS IN PLACE AND FOLLOWED FOR THE ASSESSMENT OF 
STUDENT SATISFACTION WITH ONLINE CLASSES AND PROGRAMS. 
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Figure 77 


Administrators and teachers were also asked about opportunities to assess parent satisfaction with online classes 
and programs. Once again, the majority (56.2% Administrators and 55.1% Teachers prior to COVID-19; 64.2% 
Administrators and 60.7% Teachers after COVID-19) indicated that there were processes in place to gather this 
information (Figure 78). 
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A PROCESS IS IN PLACE AND FOLLOWED FOR THE ASSESSMENT OF 
PARENT SATISFACTION WITH ONLINE CLASSES AND PROGRAMS. 


mAdministrators Teachers 
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These responses from administrators and teachers reflect a disconnect from student and parent responses. This 
could mean that more communication needs to occur regarding how students and parents can provide feedback. 


Recommendation: Provide opportunities to gather student and parent satisfaction with 
online classes and programs. 


Target Parents, Students 

Audience 

Needs Communications to students and parents regarding evaluation opportunities, their purpose, 
and how feedback is incorporated into course and program improvements 

Initiatives Create a communication plan for students and parents regarding evaluation opportunities, 
purposes, and uses 

Goals To increase awareness and opportunities for student and parent evaluations 

Actions Develop and implement a communication plan regarding evaluation opportunities, purposes, 


and uses for students and parents 

CSFs Timely and comprehensive communications are provided to students and parents regarding 
evaluation opportunities, purposes, and uses 

KPIs Student and parent participation in evaluation opportunities 


Recommendation: Communicate with parents and students about how their feedback has 
been used to improve the online learning experience. 


Target Parents, Students 
Audience 
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Needs 
Initiatives 


Goals 


Actions 


CSFs 
KPIs 
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Increased communication 

Show students and parents how their feedback has helped to improve the online learning 
courses or program 

To provide clear and consistent communication to students and parents regarding how their 
feedback has improved courses and programs 

Develop and implement a communication plan regarding how feedback has been 
incorporated 

Timely and comprehensive communications are provided to students and parents 

Student and parent satisfaction with online learning 


Subdimension: Teacher Satisfaction 


In terms of satisfaction measures for teachers, both administrators and teachers indicated high agreement that 
opportunities to collect this information existed both prior to and after COVID-19. In fact, there were 57.5% of 
administrators and 55.5% of teachers who indicated agreement that these processes were in place prior to COVID- 
19. After COVID-19, those rates went up for both administrators (64.6%) and teachers (60.5%) (Figure 79). 


A PROCESS IS IN PLACE AND FOLLOWED FOR THE ASSESSMENT OF 
TEACHER SATISFACTION WITH ONLINE CLASSES AND PROGRAMS. 
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Expert Recommendation: Consistently provide and communicate opportunities to assess 
teacher satisfaction with online classes and programs. 


Target Audience Teachers 


Needs Communications to teachers regarding evaluation opportunities, their purpose, and how 
feedback is incorporated into course and program improvements 

Initiatives Create a communication plan for teachers regarding evaluation opportunities, purposes, 
and uses 

Goals To increase awareness and opportunities for teacher evaluations 

Actions Develop and implement a communication plan regarding evaluation opportunities, 


purposes, and uses for teachers 


CSFs Timely and comprehensive communications are provided to teachers regarding evaluation 
opportunities, purposes, and uses 


KPIs _ Teacher participation in evaluation opportunities 
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Subdimension: Staff Satisfaction 


In terms of satisfaction measures for staff, both administrators and staff indicated high agreement that 
opportunities to collect this information existed both prior to and after COVID-19. In fact, there were 55.2% of 
administrators and 62.7% of staff who indicated agreement that these processes were in place prior to COVID-19. 
After COVID-19, those rates went up for both administrators (62.3%) and staff (67.4%) (Figure 80). 


A PROCESS IS IN PLACE AND FOLLOWED FOR THE ASSESSMENT OF 
STAFF SATISFACTION WITH ONLINE CLASSES AND PROGRAMS. 
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Expert Recommendation: Consistently provide and communicate opportunities to assess 
staff satisfaction with online classes and programs. 


Target Audience Staff 


Needs Communications to staff regarding evaluation opportunities, their purpose, and how 
feedback is incorporated into course and program improvements 

Initiatives Create a communication plan for staff regarding evaluation opportunities, purposes, and 
uses 

Goals To increase awareness and opportunities for staff evaluations 

Actions Develop and implement a communication plan regarding evaluation opportunities, 


purposes, and uses for staff 


CSFs Timely and comprehensive communications are provided to staff regarding evaluation 
opportunities, purposes, and uses 


KPIs Staff participation in evaluation opportunities 
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Subdimension: Innovation 


There seemed to be agreement both before and after COVID-19 in the use of innovation to guide instructional and 
programmatic decisions about online teaching and learning. Prior COVID-19, 64.3% of administrators were in 
agreement while after COVID-19, the level of agreement increased slightly to 67.9%. In fact, there was little 
disagreement by administrators with the use of innovation since only 13.7% before COVID-19 and 11.5% after 
COVID-19 indicated that they disagreed or strongly disagreed with the survey item. 


Expert Recommendation: Incorporate best practices. 


Target Administrators, Staff, Teachers 
| Audience | 
| Needs _ Access to research and international experts 
_ Initiatives Implement best practices identified by online learning experts 
| Goals | To ensure quality of online learning by benchmarking against best practices 
Actions e With due regard for national and local specifics, review quality standards (rubrics, 


scorecards, research, etc.) shared by online learning experts, groups, and/or 
organizations 
| e Apply best practices to online learning courses and programs 
CSFs _ Improved quality of online learning 
KPIs e Use of rubrics or scorecards to evaluate quality of online learning 
e Adoption of best practices 


Summary of Recommendations 


The recommendations resulting from the quantitative and qualitative analyses of this study indicated a number of 
needs and actions that will allow the Kingdom of Saudi Arabia to effectively increase and continuously evaluate 
and improve the quality, effectiveness, and capacity of K-12 online education. Recommendations are drawn from 
a wealth of research foundations and expertise, and are all focused on addressing both short- and long-term 
needs. Several expert recommendations in addition to the key actions are also provided to further support online 
education initiatives and Vision 2030. Overall findings indicate that high-need areas for online education were 
made a priority during and following COVID-19. As stated in the OECD report and document on the Saudi Arabian 
COVID perspective as well as the Ministry of Education’s Actions to Complete the School Year Amid the COVID-19 
Pandemic, K-12 education in Saudi Arabia was as well-positioned as possible at the time to respond to immediate 
and forthcoming needs. This is further confirmed by the many survey items that reflected improvements in 
perceptions and experiences with regard to online education following COVID. Study results indicated many 
opportunities to build upon the actions and initiatives that were in place prior to the pandemic and those 
implemented during, as well as new considerations to improve online K-12 education in alignment with the needs 
and goals of K-12 online education across the Kingdom. 
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Overall, study results indicate that there were many areas in which the K-12 education system was prepared 
and/or able to address quickly for online education in the COVID environment. This is potentially due to actions 
previously implemented or planned for the accomplishment of Vision 2030, and study results suggest that 
communication and understanding of such actions increased during and after COVID. The priority on increasing 
online education and Kingdom-level support for unified governance and curriculum helped schools effectively 
respond to the unexpected demands of the current environment, but because online education had not reached 
full implementation, the pandemic raised some known needs to the surface and created new needs in some areas. 
The most clear needs arose in technology (devices, internet access/reliability, and the ability of the infrastructure 
to support high-volume online education), online teaching and learning (communication, engagement, 
interaction, and feedback), assessment as related to the development and delivery of effective assessments as 
well as the need for a clear understanding of the viability and use of online assessment, and student support, as 
most of these services were previously designed for on-site usage. In continuing to evaluate and improve existing 
strengths as well as prioritizing the development and implementation of improvements for areas where gaps 
currently exist, the KSA will be able to not only progress further toward its long-range goals for online learning, but 
address educational needs related to and resulting from COVID-19. 


Recommendations are provided in detail in the narrative above and summarized in Appendix IV: Future Action 
Framework (including priority recommendations for COVID). Recommendations are primarily in support of 
furthering existing actions and addressing identified challenges related to online education. Additionally, areas 
where discrepancies are noted among stakeholder groups include recommendations to both address overall 
needs and close gaps. Primary challenges result from the unanticipated demands introduced to a system still 
undergoing planning and implementation of online education. Prior to the pandemic, the KSA strategic plan was 
in-process and steps toward building effective K-12 online education were being taken. Both during and following 
the pandemic, adjustments will need to be made to adapt to the new environment, meet immediate needs, and 
remain on track to accomplish the goals of Vision 2030. An action plan for implementing recommendations within 
the context of readiness, challenges, responses, and impact is summarized in Appendix II: Research Findings. 
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Considerations of 
Future Study 


This developmental study is comprehensive in scope and the establishment of an 
online learning evaluation framework, evaluation of pre-, peri-, and post-COVID 
contexts, practices, and needs and included both quantitative descriptive data and 
nuanced qualitative data. Resulting from the development process of the survey and 
interview protocols and evaluation frameworks, as well as from the actual study 
findings, several considerations for expanded and additional future studies are 
recommended. 


An expansion of this study into multivariate analysis could lend further exploration and indications for each 
dimension and subdimension, resulting in further recommendations for the development and implementation of 
effective practices, meeting stakeholder needs, and supporting proactive development of adequate capacity for 
current and future infrastructure, design, and delivery of online courses. Additionally, expanding on this work to 
include multivariate analyses may be valuable for the evaluation and continuous improvement of the evaluation 
framework. 


Developing and implementing longitudinal measures to follow this study is recommended for optimal evaluation 
of existing and evolving needs, and to provide clear markers of changes, successes, challenges, and opportunities. 


Additionally, consideration should be given to the development of several smaller studies designed to provide 
deep analysis of institution and stakeholder responses at more granular levels such as by school type 
(government, national, international), sector (private or public), and grade levels (e.g. primary or secondary) and 
by further disaggregated stakeholder characteristics. These analyses would benefit kingdom-level planning and 
resource allocation, as well as regional and institutional-level stakeholders by providing recommendations 
specific to their diverse environments and contexts. 


Students in the K-12 sector often need home support for successful online learning. This study indicated several 
potential challenges, including internet and device access, cultural understanding of the need for students to 
access devices and internet services, and competing priorities with regard to parental working needs and multiple 
children in the same home that need to access online learning. Case studies related to these issues, for the 
purpose of understanding challenges and opportunities to allow for more effective planning and support services 
are recommended. 
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There are a number of expansions and/or additional studies that may be beneficial to K-12 education in the KSA. 
The primary recommendations described above were selected as those most likely to provide maximum benefit 
based on the development and analysis of the present study. 
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Appendices 


Appendix I: NELC Evaluation Framework 


Governance, Strategies, Policies, Processes, Resource Allocation, Staffing, Innovation 


2 | Curriculum Design 
& Planning 
a Instructional Design Methods & UDL, Learning Objectives, Alignment, Course Materials & Content, Innovation 


Online Teaching & 
Learning 


Communication, Engagement, Expectation Setting, Outcomes, Course Interaction, Feedback, Innovation 


Assessment Strategies, Assessment Processes, Assessment Methodology, Innovation 


Centralized Online Education Infrastructure, Operability, Modality, Security, ITSM Compliance, Internet Access, 
Reliability, Coverage, Innovation 


Student Orientation & Support, Equity, Accessibility, Compliance Standards, Social Emotional, Innovation 


7 | Training & 
Support 
z Technical Assistance, Professional Development, Orientation, Mentoring, Innovation 


Evaluation & 
Continuous 
Improvement 
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Evaluation of Course Outcomes & Program Quality, Student Satisfaction, Teacher Satisfaction, Staff Satisfaction, 


Innovation 
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Appendix II: Research Methodology Framework Mapping 


Leadership 


Curriculum 
Design & 
Planning 


Baseline measures of 
subdimensions; what existing 
governance, strategies, policies, 
processes, resource availability 
and allocation, and innovations 
existed prior to COVID-19 for 
leadership? 


Objectives 


Methodology Qualitative; survey analysis and 


policy analysis 


Instruments 
(Data Source) 


Data collected via survey; 
policies provided for analysis by 
NELC 


Within each 
subdimension, what 
challenges remained, 
arose, or became more 
amplified as a result of 
COVID-19? 


Qualitative; survey 
analysis and policy 
analysis 


Data collected via survey; 


policies provided for 


analysis by NELC (if there 
has been policy change 


and/or new policy 
following COVID-19) 


Within each 
subdimension, how has 
leadership responded to 
COVID-19? 


Qualitative; survey 
analysis and policy 
analysis 

Data collected via 
survey; policies 
provided for analysis by 
NELC (if there has been 
policy change and/or 
new policy following 
COVID-19) 


Governance, Strategies, Policies, Processes, Resource Allocation, Staffing, Innovation 


Baseline measures of 
subdimensions; what 
instructional design methods, 
application of UDL, alignment 
with standards, course syllabi 
and course materials and 
content were employed prior to 
COVID-19? 

Qualitative; survey analysis and 
document review 


Objectives 


Methodology 


Within each 
subdimension, what 
challenges remained, 
arose, or became more 
amplified as a result of 
COVID-19? 


Qualitative; survey 
analysis and document 
review 


Within each 
subdimension, what has 
the response related to 
curriculum design and 
instruction been? 


Qualitative; survey 
analysis and document 
review 


Within each 
subdimension, what 
impact has occurred due 
to leadership responses 
to COVID-19? 


Qualitative; survey 
analysis and policy 
analysis 

Data collected via survey; 
policies provided for 
analysis by NELC (if there 
has been policy change 
and/or new policy 
following COVID-19) 


Within each 
subdimension, what has 
the impact of COVID-19 
on curriculum and 
instruction been? 


Qualitative; survey 
analysis and document 
review 


Recommendations 
resulting from analyses, 
both by overall 
dimension and 
subdimension. 


Qualitative; survey 
analysis and policy 
analysis 

Data collected via 
survey; policies 
provided for analysis by 
NELC 


Recommendations 
resulting from analyses, 
both by overall 
dimension and 
subdimension. 


Qualitative; survey 
analysis and document 
review 
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Online 
Teaching & 
Learning 


Instruments 
(Data Source) 


Instructional Design Methods & UDL, Learning Objectives, Alignment, Course Materials & Content, Innovation 


Objectives 


Methodology 


Instruments 
(Data Source) 


Data collected via survey; 
documents for review requested 
as part of survey instrument 


Baseline measures of 
subdimensions; what methods 
of online course communication, 
engagement, expectation 
setting, outcomes, course 
interaction, feedback, and 
resources were employed prior 
to COVID-19? 

Mixed methods; survey analysis, 
document review, data analysis 


Data collected via survey; 
documents for review and data 
for analysis requested as part of 
survey instrument 


Data collected via survey; 


documents for review 
requested as part of 
survey instrument 


Within each 
subdimension, what 
challenges remained, 
arose, or became more 
amplified as a result of 
COVID-19? 


Mixed methods; survey 
analysis, document 
review, data analysis 


Data collected via survey; 


documents for review 
and data for analysis 
requested as part of 
survey instrument 


Data collected via 
survey; documents for 


review requested as part 


of survey instrument 


Within each 
subdimension, what has 
the response related to 
online teaching and 
learning been? 


Mixed methods; survey 
analysis, document 
review, data analysis 


Data collected via 
survey; documents for 
review and data for 
analysis requested as 
part of survey 
instrument 


Communication, Engagement, Expectation Setting, Outcomes, Course Interaction, Feedback, Innovation 


Data collected via survey; 
documents for review 
requested as part of 
survey instrument 


Within each 
subdimension, what 
impact has occurred due 
to online teaching and 
learning responses to 
COVID-19? 


Mixed methods; survey 
analysis, document 
review, data analysis 


Data collected via survey; 
documents for review 
and data for analysis 
requested as part of 
survey instrument 


Data collected via 
survey; documents for 
review requested as part 
of survey instrument 


Recommendations 
resulting from analyses, 
both by overall 
dimension and 
subdimension. 


Mixed methods; survey 
analysis, document 
review, data analysis 


Data collected via 
survey; documents for 
review and data for 
analysis requested as 
part of survey 
instrument 


135 


4 


5 


OL Cel Beeresre 


Assessment 


Technology 
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Objectives 


Methodology 


Instruments 
(Data Source) 


Baseline measures of 
subdimensions; what 
assessment strategies, 
processes, methodologies, and 
types were employed prior to 
COVID-19? 

Mixed methods; survey analysis, 
document review, data analysis 


Data collected via survey; 
documents for review and data 
for analysis requested as part of 
survey instrument 


Within each 
subdimension, what 
challenges remained, 
arose, or became more 
amplified as a result of 
COVID-19? 

Mixed methods; survey 
analysis, document 
review, data analysis 


Data collected via survey; 
documents for review 
and data for analysis 
requested as part of 
survey instrument 


Within each 
subdimension, what has 
the response related to 
assessment been? 


Mixed methods; survey 
analysis, document 
review, data analysis 


Data collected via 
survey; documents for 
review and data for 
analysis requested as 
part of survey 
instrument 


Assessment Strategies, Assessment Processes, Assessment Methodology, Innovation 


Objectives 


Methodology 


Instruments 
(Data Source) 


Baseline measures of 
subdimensions; what was 
present and used regarding 
operability, centralized online 
education infrastructure, 
modality, security, ITSM 
Compliance, internet access, 
reliability, and coverage prior to 
COVID-19? 

Qualitative; survey analysis 


Data collected via survey 


Within each 
subdimension, what 
challenges remained, 
arose, or became more 
amplified as a result of 
COVID-19? 


Qualitative; survey 
analysis 


Data collected via survey 


Within each 
subdimension, what has 
the response related to 
technology been? 


Qualitative; survey 
analysis 


Data collected via 
survey 


Within each 
subdimension, what 
impact has occurred due 
to online assessment 
responses to COVID-19? 


Mixed methods; survey 
analysis, document 
review, data analysis 


Data collected via survey; 
documents for review 
and data for analysis 
requested as part of 
survey instrument 


Within each 
subdimension, what 
impact has occurred due 
to technology responses 
to COVID-19? 


Qualitative; survey 
analysis 


Data collected via survey 


Recommendations 
resulting from analyses, 
both by overall 
dimension and 
subdimension. 


Mixed methods; survey 
analysis, document 
review, data analysis 


Data collected via 
survey; documents for 
review and data for 
analysis requested as 
part of survey 
instrument 


Recommendations 
resulting from analyses, 
both by overall 
dimension and 
subdimension. 


Qualitative; survey 
analysis 


Data collected via 
survey 
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Student 
Support 


Training & 
Support 


Centralized Online Education Infrastructure, Operability, Modality, Security, ITSM Compliance, Internet Access, Reliability, Coverage, Innovation 


Objectives 


Methodology 


Instruments 
(Data Source) 


Baseline measures of 


subdimensions; what was the 
availability and use of student 
orientation and support services, 


equity, accessibility, and 


compliance prior to COVID-19? 
Mixed methods; survey analysis, 
document review, and data 


analysis 


Data collected via survey; 


documents for review and data 
for analysis requested as part of 


survey instrument 


Within each 
subdimension, what 
challenges remained, 
arose, or became more 
amplified as a result of 
COVID-19? 

Mixed methods; survey 
analysis, document 
review, and data analysis 


Data collected via survey; 
documents for review 
and data for analysis 
requested as part of 
survey instrument 


Within each 
subdimension, what has 
the response related to 
student support been? 


Mixed methods; survey 
analysis, document 
review, and data 
analysis 


Data collected via 
survey; documents for 
review and data for 
analysis requested as 
part of survey 
instrument 


Student Orientation & Support, Equity, Accessibility, Compliance Standards, Social Emotional, Innovation 


Objectives 


Methodology 


Baseline measures of 


subdimensions; what was the 
availability and use of technical 


assistance, professional 


development, orientation, 
mentoring, and/or other support 


prior to COVID-19? 


Mixed methods; survey analysis 


and data analysis 


Within each 
subdimension, what 
challenges remained, 
arose, or became more 
amplified as a result of 
COVID-19? 


Mixed methods; survey 
analysis and data 
analysis 


Within each 
subdimension, what has 
the response related to 
training and support 
been? 


Mixed methods; survey 
analysis and data 
analysis 


Within each 
subdimension, what 
impact has occurred due 
to online student support 
needs and responses to 
COVID-19? 

Mixed methods; survey 
analysis, document 
review, and data analysis 


Data collected via survey; 
documents for review 
and data for analysis 
requested as part of 
survey instrument 


Within each 
subdimension, what 
impact has occurred due 
to training and support 
responses to COVID-19? 


Mixed methods; survey 
analysis and data 
analysis 


Recommendations 
resulting from analyses, 
both by overall 
dimension and 
subdimension. 


Mixed methods; survey 
analysis, document 
review, and data 
analysis 


Data collected via 
survey; documents for 
review and data for 
analysis requested as 
part of survey 
instrument 


Recommendations 
resulting from analyses, 
both by overall 
dimension and 
subdimension. 


Mixed methods; survey 
analysis and data 
analysis 
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Instruments 
(Data Source) 


Technical Assistance, Professional Development, Orientation, Mentoring, Innovation 


Objectives 


Methodology 


Instruments 
(Data Source) 


Data collected via survey; 
documents for review and data 
for analysis requested as part of 
survey instrument 


Baseline measures of each 
subdimension; levels of student, 
teacher, and staff satisfaction, 
evaluation of course outcomes, 
and program quality prior to 
COVID-19. 

Mixed methods; survey analysis, 
document review, data analysis 


Data collected via survey; 
documents for review and data 
for analysis requested as part of 
survey instrument 


Data collected via survey; 
documents for review 
and data for analysis 
requested as part of 
survey instrument 


Within each 
subdimension, what 
challenges remained, 
arose, or became more 
amplified as a result of 
COVID-19? 

Mixed methods; survey 
analysis, document 
review, data analysis 


Data collected via survey; 
documents for review 
and data for analysis 
requested as part of 
survey instrument 


Data collected via 
survey; documents for 
review and data for 
analysis requested as 
part of survey 
instrument 


Within each 
subdimension, what has 
the response related to 
evaluation and 
continuous 
improvement been? 
Mixed methods; survey 
analysis, document 
review, data analysis 


Data collected via 
survey; documents for 
review and data for 
analysis requested as 
part of survey 
instrument 


Data collected via survey; 
documents for review 
and data for analysis 
requested as part of 
survey instrument 


Within each 
subdimension, what 
impact has occurred due 
to evaluation and 
continuous improvement 
responses to COVID-19? 
Mixed methods; survey 
analysis, document 
review, data analysis 


Data collected via survey; 
documents for review 
and data for analysis 
requested as part of 
survey instrument 


Evaluation of Course Outcomes & Program Quality, Student Satisfaction, Teacher Satisfaction, Staff Satisfaction, Innovation 


Data collected via 
survey; documents for 
review and data for 
analysis requested as 
part of survey 
instrument 


Recommendations 
resulting from analyses, 
both by overall 
dimension and 
subdimension. 


Mixed methods; survey 
analysis, document 
review, data analysis 


Data collected via 
survey; documents for 
review and data for 
analysis requested as 
part of survey 
instrument 


Research Findings 


Subdimensions During Covid 19 Post Covid 19 


Governance Appropriate governance Timely communication and |Increased communications — |Increased satisfaction, Share and communication 
Leadership structures were in place decisions and inclusion of teachers and | awareness, and decision-making rationale and 
staff in decision-making understanding of processes with local school 


governance structures and |stakeholders 
decisions 


Strategies Overall satisfaction with Aligning strategies for online | Increased Increased satisfaction with | Create alignment with and between 
strategic planning and learning with other communication and online learning strategies | the institution’s strategic plan for 
alignment strategies (i.e. academic, development of plans and identification of gaps | online learning with other strategic 

technology) potentially occurred with other plans planning components 


Policies Policies broadly Implementation of policies | Policies have been reviewed | Increased satisfaction with | Explore policies that aim to improve 
supported the effective regarding academic and updated to encourage policies directly relevant to | quality in online teaching, learning, 
development and delivery of integrity, online teaching online learning and address | and supporting online and innovation 


online education and learning, and issues of academic integrity, |education 
innovation online teaching and learning, 
and innovation 


Process Affirmation that processes |Clarityand implementation |Increased communication Increased agreement that |Improve clarity in processes for 
were in place to support of processes related to and implementation of effective processes arein evaluation 
online education needs, teacher evaluations and processes and potential place 


though some needs were resource allocations development or revision of 
identified processes during COVID 


Resource Majority agreement on Uneven distribution of Resource review and Increased agreement that |Explore the extent to which 
Allocation support and efforts, though |resources overall and a need | provision to sustain and scale |resource allocations are resources are distributed equitably 
uneven allocations were for more professional online learning sufficient to support needs; | across schools; seek opportunities 
noted development resources uneven distribution to address resource allocation 
remains a concern related to incentivizing or providing 
financial compensation for teacher 
training and instructional design 
professional learning 


OL Rae 


Curriculum 
Design & 
Planning 


Staffing 


Instructional 
Design Methods & 
UDL 


Learning 
Objectives 


Majority agreement that 
sufficient qualified, 
professional, administrative, 
support, and technical staff 
were available to support 
online learning 


Majority agreement that 
innovation is supported 


Instructional design and 
UDL was sufficient to meet 
pre-COVID needs, but the 
pandemic revealed 
pedagogical considerations 
that may not have been 
originally included in the 
curriculum 


A development process was 
followed to ensure that 
online classes are designed 
for students to develop the 
necessary knowledge and 
skills to meet measurable 
learning objectives 


There were guidelines 
regarding standards for 
class development, 
supported by alignment 
with instructional design 
and delivery 


Additional staffing resources | Staffing was provided across 


are needed to adequately 
address online learning in 
the fully-online COVID 
environment 


Determine the role of 
innovation in staff 
evaluations and promotions 


Improvement of design to 
address new or newly- 
identified needs such as 
maintaining student 
relationships and 
addressing social-emotional 
needs in the online 
environment 


Engagement of students in 
building meaningful 
understanding and intrinsic 
motivation to achieve 
learning objectives 


Aligning content and 
learning activities with 
assessments and learning 
objectives 


a variety of roles to support 
fully-online learning, though 
a need for additional 
resources remains 


No explicit actions were 
indicated relevant to 
alignment 


Instructional design support 
from teachers, instructional 
designers, technical experts, 
and evaluators as well as the 
Ministry of Education 


No explicit actions were 
indicated relevant to learning 
outcomes 


No explicit actions were 
indicated relevant to 
alignment 


Increased satisfaction with 
available staffing 
resources; stated 
appreciation in interviews 
affirming an increase in 
needs being met 


Increase in agreement that 
innovation is supported 


Satisfaction decreased, 
indicating that while 
additional resources were 
provided, they may not 
have been able to fully 
address unanticipated 
needs resulting from the 
pandemic 


Agreement increased 
regarding the presence of 
measurable learning 
objectives, suggesting that 
the pandemic required 
closer attention to and/or 
engagement with existing 
processes 


Slight decrease in 
agreement that content 
and learning activities are 
aligned with assessments 
and learning objectives, 
suggesting difficulty with 
unique challenges resulting 
from COVID-19 


For leadership to continue to 
pursue opportunities and 
incentives for innovation and 
outstanding service provision on 
the part of staff members 


For leadership to continue to 
pursue opportunities and 
incentives for innovation and 
outstanding service provision on 
the part of staff members 


Utilize and articulate the 
instructional design decisions, 
frameworks, and philosophies that 
guide the K-12 curriculum design 
across the Kingdom; Add 
pedagogical and design context 
around course materials 


For teachers to engage students in 

building meaningful understanding 
and intrinsic motivation to achieve 
learning outcomes 


For teachers to incorporate 
supports to ensure that standards- 
based curriculum and instruction is 
aligned to learning outcomes, as 
well as being accessible and 
understandable to each student 


Course Materials 
& Content 


Innovation 


Communication 


Engagement 


Expectation 
Setting 


Current, sufficient, and rich 
content, both instructional 
and operational/functional, 
was present in classes 


Majority agreement that 
innovation was supported, 
though some indicated that 
this may be localized by 
school 


Majority agreement that 
opportunities for a variety of 
communication types were 
present and expectations 
were in place 


Classroom environments 
were both academically 
challenging and provided 
social interaction 


Clear, concise, and useful 
assignment instructions as 
well as expectations for 
participation and feedback 
were provided 


Time and skill needed to 
continually create engaging 
content 


Broad and equal support for 
innovation across all schools 


Effectively communicating 
ina fully-online 
environment 


Identification and use of 
communication methods to 
support and encourage 
student engagement 


Ensuring clarity and 
communication of 
expectations in a fully-online 
environment 


A variety of materials, 
including interactives and 
OER are available; the 
Ministry provides a variety of 
content sources 


Potential increases in support 


for innovation in online 
education 


Changes made to better 
support students via 
communications (frequency 
and/or method) during 
transition and while fully- 
online; increased adoption of 
expectations 


Teachers independently 
created strategies for 
providing engagement 
opportunities and 
communicating directly with 
students to support 
engagement 


Teacher indications that they 


eliminated barriers to success 


by providing clear 


Slight increases in 
agreement that these items 
are present in online 
courses, with the 
acknowledgment that 
maintenance will require 
time and skill 


Slight increase in 
agreement that innovation 
is supported 


Increase in satisfaction 
with communication 
opportunities and 
effectiveness 


Increase in satisfaction that 
engagement opportunities 
are present and effective 


Increase in agreement that 
clear expectations are 
provided throughout 
expectations for participation | online courses 

and performance 


For the Ministry of Education 
regularly review course materials 
and content for sufficient coverage 
and currency, and update as 
appropriate with current and 
media-rich content 


For teachers to expand their efforts 
to innovate and redesign their 
curriculum 


Develop processes and policies for 
expectations around 
communications 


Provide engagement opportunities, 
both formal and informal, for 
students to interact with one 
another and teachers, and ensure 
that academic challenge is 
incorporated into these experiences 
as appropriate 


Ensure that online courses include 
clearly defined student activity 
expectations, and that these 
expectations are communicated to 
students 
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Outcomes 


Course 
Interaction 


Feedback 


Innovation 


Assessment 
Strategies 


Outcomes were sometimes, 
but not always, comparable 
between on-site and online 
courses 


Student-to-student and 
teacher-to-student 
interaction opportunities in 
a variety of formats were 
provided and encouraged 


Teachers provided students 
with constructive, timely 
feedback 


Majority agreement that 
innovation was encouraged 


Most participants agreed or 
were neutral with regard to 
whether assessment 

strategies were satisfactory 


Equivalence in outcomes 
between on-site and online 
courses; alignment between 
outcomes, objectives, 
learning activities, and 
assessments 


Synchronous 
communication barriers 
present a challenge to the 
effective provision of 
interaction opportunities 


Difficulty keeping up with 
expectations for feedback 
while also meeting student 
needs resulting from the 
rapid transition to remote 
learning 


Potential challenges in 
shifting innovation focus to 
the online environment 


Adoption of a variety of 
assessment strategies in the 
online environment 


Outcomes for online courses 
as well as alignment of 


outcomes potentially revised 


for more congruence in the 
online environment 


Levels of communication 
created enough of a 
challenge that interactions 
suffered 


Feedback may have been less 


frequent or timely due to the 
rapid transition and shifting 
needs resulting from the 
pandemic 


Encouragement and support 
for innovation increased; 
evidence of a strong 
willingness by teachers and 
staff to engage with 
innovation 


Assessment strategies were 
piloted to address learning 


needs as well as to implement 
diverse formative assessment 


strategies 


Increases in agreement 
that both congruence in 
outcomes increased as well 
as alignment of outcomes 
with other course 
components 


Decrease in the agreement 
that interaction 
opportunities are present 
in online courses 


Decrease in agreement that 
constructive, timely 
feedback is provided 


Increase in agreement that 
innovation is encouraged 


Slight increases in 
agreement that 
assessment strategies are 
satisfactory; neutral 
responses consistent 


Measure, review, and communicate 
student learning outcomes for 
online courses, and if appropriate 
and/or useful, include 
communicating comparisons to 
student learning outcomes for 
onsite courses; Provide rubrics for 
all assignments 


Encourage opportunities for 
teacher-to-student and student-to- 
student interaction in the online 
classroom; Establish the presence 
of the online educator 


Set expectations for feedback 
provision and methods, as well as 
when feedback will be provided and 
how it is related to course learning 
objectives 


Consistently provide and encourage 
opportunities for innovation in 
online teaching and support of 
students 


Reduce the existing gaps between 
school administrators, staff, 
teachers, and students in 
communication, expectations, and 
implementation of assessment 
strategies, processes, 
methodologies, as well as types of 
assessment 


Technology 


Assessment 
Processes 


Assessment 
Methodology 


Innovation 


Centralized 
Online Education 
Infrastructure 
(SSO, LMS, etc.,) 


Majority agreement that 
effective assessment 
processes were in place 
including alignment and 
communication of 
assessment criteria, grading, 
expectations, and feedback 


Majority agreement that 
assessment methods were 
satisfactory 


General satisfaction with the 
level and support of 
innovation 


Majority agreement that 
centralized online education 
infrastructure was present 
and sufficient 


Online implementation and 
ensuring student 
understanding of 
assessment procedures in 
the online environment 


Opportunities and ability to 
create diverse online 
assessments and concern 
about perceptions of the 
viability of peer- and self- 
assessment 


Encouragement and 
sufficient training on the 
conceptualization, design, 
and delivery of assessment 
innovations 


Centralization is not 
supported at a level 
sufficient to manage the 
increase in needs due to 
COVID-19 


No explicit actions were 
indicated relevant to 
assessment processes 


No explicit actions were 
indicated relevant to 
assessment methodology 


No explicit actions were 
indicated relevant to 
assessment innovation, 
though teacher enthusiasm 
for the integration of 
advanced technology, 
platforms, instruments, and 
tools was high 


No explicit actions were 
indicated relevant to 
centralized education 
infrastructure 


Decrease in agreement and 
increase in neutral 
responses and 
disagreement that effective 
assessment processes are 
in place 


Decrease in agreement and 
increase in both neutral 
responses and 
disagreement that 
assessment methods are 
satisfactory 


Decrease in satisfaction 
with support for innovation 


Increase in agreement and 
decrease in disagreement 
that centralized education 
infrastructure is present 
and sufficient 


Ensure alignment of students’ 
learning objectives and 
assessments through transparent 
quality review processes; Clearly 
describe assessment criteria when 
assignments are issued, provide 
grading rubrics and samples of 
assignments illustrating teacher/ 
school/ system expectations 


Provide students opportunities to 
demonstrate their subject 
proficiency in various ways; 
Diversify formative and summative 
assessment 


Innovation in assessment 
strategies, processes, methods, and 
types is encouraged, supported, 
and actualized; adequate and 
consistent professional 
development on design, 
development, and delivery is 
provided 


Provide a centralized learning 
system with single sign-on (SSO) or 
single identification. Focus should 
prioritize schools that have not yet 
implemented online learning as 
supported by the Future Gate 
initiative, as these are the most 
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likely to indicate absence of 
centralized solutions 


Operability Majority agreement that Parent and student No explicit actions were Perceptions of agreement |Given the disconnect between the 
technology operability was |perceptions of operability indicated relevant to remained consistent with |educators and parents and 
sufficient are less favorable than operability pre-COVID responses students, schools can develop 

administrators, staff, and feedback channels and provide 
teachers orientation and training around 
learning technologies 


Modality Majority agreement that Availability of devices and Potential improvements to Increase in agreement in Evaluate the student experience in 
online classes are delivered |internet service sufficient to |modality of course offerings |suitable modality use synchronous and asynchronous 
asynchronously and/or support asynchronous and | necessitated by the pandemic} depending on learning online learning in order to improve 


synchronously dependent __|synchronous functions of outcomes course design and outcomes 
on learning outcomes online courses 


Security Majority agreement that Need for teaching students |Individualschoolorteacher Perceptions of online Greater transparency and clear 
digital security measures are | to understand internet efforts to increase awareness |security remain stable and | policies and guidelines can increase 
in place to ensure the security and protect and safe practices related to |similarto pre-COVID rates |stakeholder trust and buy-in 
integrity and validity of themselves; lower parent online security among of 
information, including agreement than for other students agreement/disagreement 
identity access groups 


ITSM Compliance | Majority agreement that Despite agreement by most, | No explicit actions were Increase in agreement and |Administrators should evaluate 
ITSM designed, delivered, |many disagreed thatITSM _ |indicated relevant to ITSM decrease in disagreement |current processes, plans, and 
operated, and controlled support is sufficient compliance that ITSM support is resources that are available and 
information supporting sufficient, potentially due |where gaps or inefficiencies exist 
online course delivery and to increased use and 
development awareness 


Internet Access {Availability of adequate Equitable access to Some schools andteachers Increase in agreement and |For students who have poor or no 
internet access and schools |sufficient internet to support] made independent efforts to |decreasein disagreement |internet access at home, schools 
and homes was widely online learning increase availability of regarding the availability of | can provide mobile hotspots and 
variable devices and internet access _|internet access; however, |develop content that requires 

this remains a top minimum bandwidth 


challenge for effective 
online education 


Decrease in agreement that | Monitor the reliability of learning 


Reliability 


students 


Coverage 


Innovation Majority agreement that 
new technologies were 
piloted, supported, and 
encouraged in online classes 


Student 
Orientation & 
Support 


issues Support 


Coverage was inconsistent 
among respondents 


Clear discrepancies between 
administrators/staff/teacher 
s and parents/students 
about the adequate 
availability of orientation, 
academic and technical 
support, and interpersonal 


Reliability of technology was | Ensuring consistent 
varied among schools and 


reliability of core learning 
technology systems and 
courses due to support 
needs and network load 


Providing consistent and 
equitable coverage across 
the country 


Specific challenges are not 
identified for technology 
innovation 


Online orientation and 
support was largely 
unavailable prior to COVID; 
Students indicated high 
need for more academic 
support 


No explicit actions were 
indicated relevant to 
reliability 


No explicit actions were 
indicated relevant to 
coverage 


Encouragement and support 


for innovation potentially 
increased 


No explicit actions were 


indicated relevant to student 


orientation and support, 
though some individual 
examples were provided 


technology is sufficiently 
reliable to support current 
online education needs 


Indication that coverage 
improvements remain 
necessary 


Slight increase in 
agreement and decrease in 
disagreement that 
technological innovation is 
encouraged and supported 


Consistent with pre-COVID 
perceptions, except for 
academic support among 
students, which decreased 
in agreement and 
increased in disagreement 


technology systems and courses as 
related to infrastructure (i.e. 
challenges resulting from 
infrastructure, not from individual 
internet access, etc.); Identify and 
address areas needing system-level 
solutions 


Given the need for increased 
learning technologies coverage, a 
plan that includes monitoring and 
feedback loops to continually 
determine need, as well as 
mechanisms to provide access to 
technologies is critical 


With the significant increase in data 
collection through online tool use, 
teachers and administrators have 
increased potential to improve 
student outcomes through real- 
time, actionable insights to inform 
interventions 


Provide mandatory orientation to 
online courses, including 
technology, class design, academic 
support resources, and 
technological support resources; 
Ensure that academic support 
resources are available and may be 
accessed online 


Ou & ONLINE LEARNING 
CONSORTIUM 


Slight majority agreement |The online environment Some schools andteachers | Slight increase in Develop and implement measures 

that resources were in place |presents unique challenges |made independent efforts to |agreement and decrease in |to evaluate student support needs 

to ensure all students have related to equity, including |increase equity disagreement that related to equity, including 

an equitable experience identifying needs and resources to provide equity | financial, health, and academic 

providing support services are in place areas; Based on evaluations of 

student support needs, develop 
solutions to provide student 
support that may be accessed by 
students either digitally, or that 
account for the inability for 
students to be on-site 


Accessibility Accessibility efforts Providing adequate No explicit actions were Accessibility efforts reflect | Ensure that online courses follow 
including policies, processes | accessibility online following | indicated relevant to responses consistent with | accessibility standards 
and available resources the rapid shift due to COVID- | coverage pre-COVID perceptions 
were primarily concentrated | 19 
on on-site courses 


Compliance Slight majority agreement |Ensuringthatcompliance |No explicit actions were Increase in agreement and |Ensure that accessibility standards 
Standards that online classes standards are met in the indicated relevant to decrease in disagreement |are in alignment with compliance 
demonstrate compliance online environment compliance standards that compliance standards |standards 
with accessibility standards are met in online courses 


Social Emotional | Majority agreement that Providing social-emotional |Someschoolsandteachers | Slight increase in Develop and implement measures 
students were provided with | support in the online made independent efforts to |agreement that social- to evaluate student support needs 
opportunities for social- environment, particularly provide social-emotional emotional learning and related to social emotional areas; 
emotional learning and with new or changed needs _ |support services in the online |support are provided Based on evaluations of social 
support resulting from COVID-19 and | environment emotional student support needs, 

as related to support for develop solutions to provide 

student well-being support that may be accessed by 
students either digitally, or that 
account for the inability for 
students to be on-site; Provide 
course-based opportunities for the 
social emotional support of 
students 


Innovation 


Technical 
Assistance 


Training & 
Support 


Professional 
Development 


Orientation 
(Student & 
Parent) 


Mentoring 


Majority agreement that 
innovation for student 
support communication 
modes and tools was 
present 


Majority satisfaction with 
technical assistance 
provision for online class 
development and teaching 


Majority agreement that 
teachers receive training 
assistance and support to 
prepare for course 
development and remote 
education 


Majority agreement that 
sufficient orientation to 
technology and instructional 
design was present 


Some peer mentoring 
opportunities were available 
to teachers 


Discovery and 
implementation of online 


student support services 


Teachers face a variety of 
technical challenges related 
to the effective use and 
support of technology 


Providing adequate 
professional development 
amid multiple priorities 
accelerated by the 
pandemic 


Developing and maintaining 
relevant orientation amid 
rapidly changing 
circumstances due to 
COVID-19 


Uneven opportunities for 
teachers to participate in 
peer mentoring activities 


No explicit actions were 
indicated relevant to 


innovation in student support 


No explicit actions were 
indicated relevant to 
technical assistance; 
however, outcomes indicate 
increased availability and/or 
use of services 


Potential changes to the 
delivery and relevant topics 
covered by professional 
development as a result of 
the pandemic 


Potential increase in the 
development and availability 
of orientation to technology 
and instructional design 


No explicit actions were 
indicated relevant to peer 
mentoring; however, the 
circumstances of COVID-19 
may have encouraged formal 
or informal engagement in 
such opportunities 


Slight increase in 
agreement, potentially 
related to necessary 
development of student 
support services in the 
online environment 


Increase in agreement and 
decrease in disagreement 
that technical assistance is 
satisfactory to support 
development and teaching 
for online classes 


Increase in agreement, and 
particularly in strong 
agreement that adequate 
professional development 
opportunities are provided 


Increase in agreement that 
sufficient orientation is 
provided 


Increase in agreement that 
peer mentoring 
opportunities are available 


Include innovative communication 
modes and tools for supporting 
students in online environments 


Develop technology-facilitated 
web-mediated guidance and 
instruction program to integrate 
principles for meaningful and 
sustained online learning in 
addition to addressing workplace 
barriers to sustained learning 
through non-stop training; 
Feedback from all members of the 
education process is a pre- 
condition for ensuring appropriate 
technical assistance 


Ensure ongoing professional 
development, training and 
assistance in pedagogy and 
technology; Move long, 
synchronous trainings into 
modular, asynchronous formats 


Provide abundant, comprehensive 
and sufficient pedagogical and 
technology-specific orientation 


Acentrally-coordinated virtual 
mentoring program can ensure 
opportunities for more consistent 
and viable connection and support 
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Evaluation & 
Continuous 
Improvement 


Innovation 


Student 
Satisfaction 


Teacher 
Satisfaction 


Staff Satisfaction 


Majority agreement that 
there was support for 
selecting new tools and 
piloting emerging 
technologies 


Disagreement among 
parents and students that 
students were provided with 
an opportunity to provide 
course feedback and that 
they were informed about 
the outcomes of such 
feedback 


Majority agreement that 
opportunities to collect 
teacher satisfaction 
evaluations were available 


Majority agreement that 
opportunities to collect staff 
satisfaction evaluations 
were available 


Balancing immediate online 
learning needs with piloting 
innovative opportunities 


Improved communication is 
needed regarding how 
students and parents 
provide feedback, and how 
that feedback is used 


Frequency and clarity of 
information to teachers 
regarding satisfaction 
measures and feedback 
opportunities 


Frequency and clarity of 
information to staff 
regarding satisfaction 
measures and feedback 


opportunities 


No explicit actions were 
indicated relevant to 
innovation 


No explicit actions were 
indicated relevant to student 
satisfaction 


No explicit actions were 
indicated relevant to teacher 
satisfaction; however, it is 
possible that additional 
communications, 
opportunities, and/or 
increased participation in 
such opportunities occurred 


No explicit actions were 
indicated relevant to staff 
satisfaction; however, it is 
possible that additional 
communications, 
opportunities, and/or 
increased participation in 
such opportunities occurred 


Increase in agreement and 
decrease in disagreement 
that innovation is 
supported 


Consistent rates of 
agreement and 
disagreement both before 
and after COVID-19 


Increase in agreement that 
opportunities to assess 
teacher satisfaction are 
provided 


Increase in agreement that 
opportunities to assess 
staff satisfaction are 
provided 


Support awareness and use of new 
technologies through a teacher 
learning community program 


Provide opportunities to gather 
student and parent satisfaction 
with online classes and programs; 
Communicate with parents and 
students about how their feedback 
has been used to improve the 
online learning experience 


Consistently provide and 
communicate opportunities to 
assess teacher satisfaction with 
online classes and programs 


Consistently provide and 
communicate opportunities to 
assess Staff satisfaction with online 
classes and programs 


Evaluation of 
Course Outcomes 
and Program 
Quality 


Innovation 


Majority agreement that 
procedures and policies for 
evaluating and improving 
online courses and 
programs were in place 


Majority agreement that 
innovation is used to guide 
instructional and 
programmatic decisions 
regarding online education 


Lack of consistent or formal 
processes, as well as 
availability and use of data 
for evaluation and 
improvement 


No explicit challenges 
indicated 


Steps may have been taken to 
improve processes as 
reflected by increases in 
agreement 


No explicit actions were 
indicated relevant to 
innovation 


Increase in agreement and | Provide processes, procedures, and 
identification of aneed for | policies on course and program 
clear procedures to be evaluation, including regular review 
provided by the Ministry of |and updating 

Education 


Increase in agreement and _|Incorporate best practices 
decrease in disagreement 

that innovation guides 

instructional and 

programmatic decisions 

regarding online education 
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Appendix III: Terminology 


This appendix contains the most common current terms used to describe various iterations of learning 
via remote, distance, and online means. Below are commonly accepted terminology definitions, but it 
should be noted that they often overlap and even contradict one another. Guidelines for selecting terms 
to be used institutionally or on a larger scale, such as system- or kingdom-wide can be found in the 
Terminology section of Appendix VII: Literature Review. 
= Emergency Remote Teaching: A temporary shift of instructional delivery from its original 
modality to one that fits the current situation due to crisis situations. 
" Online Education: Courses are designed, developed, and implemented in a fully online 
environment. Courses may be synchronous, asynchronous, or a combination. 
" Distance Education: Teaching and learning is provided at a distance and may be offered 
through various means such as online, video, correspondence, etc. 
= eLearning: Teaching and learning is provided through the use of electronic technologies. 
"  Hybrid/Blended Courses: Courses are provided in a format blending multiple modalities. Most 
often, some face-to-face components are combined with online and/or distance education. 
= HyFlex Courses: Courses are designed and offered in a flexible format allowing for face-to-face, 
online, or hybrid/blended modalities that may be selected based on need or preference. 


Embedded in these terms used to describe overall variations in online learning are several terms 
describing common practices. 

= Synchronous: The learner and instructor are online at the same time for synchronous 
instructional delivery, course activities, and/or interaction and engagement. 

" Asynchronous: Content delivery, course activities, and opportunities for engagement and 
interaction may be completed independently or at different times for each instructor and 
learner. 

" Online Collaborative Learning: The use of synchronous, asynchronous, or mixed methods for 
course delivery and completion 

= Computer-Based Training: The provision of modular online learning, often in a self-paced 
format. This type of online learning is typically used in continuing education or professional 
development. 

« Digital Education, or Technology-Enhanced Learning: The use of digital technologies to 
provide education, including systems (i.e. Learning Management Systems), tools (i.e. 
applications for communication or learning activities), or course artifacts (i.e. digital activities 
and interactions). 


There are also a number of course-level terms that are essential for development and delivery of online 
learning. 
« Instructional Design: The systematic process of designing, developing, and delivering effective 
instruction through effective structure and presentation of learning materials and content, 
course activities and interactions, and assessment. 
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Universal Design for Learning (UDL): A framework to improve and optimize teaching and 
learning for all individuals. 

Learning Outcomes: What the learners will demonstrably know and/or be able to do at the end 
of a course. 

Learning Objectives: What the learners will demonstrably know and/or be able to do as a result 
of individual units or activities. 

Course Materials and Content: Learning content and supporting materials designed to support 
learning outcomes and objectives. 

Communication: Provision and exchange of information, including email, discussions, 
synchronous meetings, documents, etc. in a course. 

Engagement: The level of student thinking, communication, and interaction with course 
content, materials, activities, and participants. 

Course Interaction: Active student engagement with course components and participations. 
Generally considered in three forms. 

e Student-Content Interaction: Active student engagement with learning content, 
materials, activities, and assessments. 

e Student-Student Interaction: Active student engagement with other students, 
including video or discussion, group projects, collaborative work, etc. 

e Student-Instructor Interaction: Active student engagement with instructors, 
including communications, discussions, feedback, participation through course 
activities, etc. 

Feedback: Communication about performance (or experience) that is provided and 
subsequently used to increase understanding and/or improve outcomes. 
Assessment: The evaluation, measurement, and documentation of learning and/or 
achievement of outcomes and objectives. 
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Appendix IV: NELC Future Action Framework for K-12 


Leadership 


152 


Subdimensions 


Governance 


Strategies 


Frequent and 
timely 
communication 
with local school 
stakeholders 


Articulation of and 
resource allocation 
for strategic 
alignment between 
online learning 
plans and other 
strategic planning 
components 


NELC FUTURE ELEARNING ACTION FRAMEWORK - K-12 


Initiatives 


Provide specific and 
timely 
communication 
related to fluidity 
with COVID and 
online learning 


Comprehensive 
strategic planning 
to account for any 
discrepancies or 
disparities between 
planning goals and 
organizational 
alignment 


To communicate 
decision-making 
rationale and 
processes with 
local school 
stakeholders 


To articulate a 
coherent, 
congruent 
strategic plan that 
accounts for 
online learning in 
a way that aligns 
with other 
priorities and 
plans 


Critical Success 


Action Lin 
Suomi ines Factors 


- Identify existing 
communication 
strategies and channels 
- Explore possibilities 
for ongoing 
communication with 
school-level 
stakeholders - Solicit 
feedback from school- 
level stakeholders - 
Evaluate effectiveness 
of communications 

- Review current 
strategic planning 
across the organization 
(including but not 
limited to online 
learning, technology, 
and assessment) - 
(Re)Establish strategic 
planning team and 
schedule regular 
reviews for 
congruences and 
discrepancies across 
strategic planning 
efforts - Establish 
congruent and aligned 
strategic plan, 
accounting for online 
learning 


Schools receive 
timely 
communication as 
COVID fluidity may 
influence decision- 
making 


Sensemaking and 
synthesizing 
planning at various 
school sites and 
contexts 


Priority 
for 
COVID 


Target Audience 


- Frequency of 

regular 

communication 

channels and 

flows - Percentage | Administrators 
of resources 

aligned to 

communication 

channels 


- Strategic plan 
action items are 


aligned with short- 
and long-term 
goals - Strategic 
plan is 
communicated to 
relevant 
stakeholders 


Administrators 


- Review existing 
policies related to 
online teaching, 
learning, academic 
integrity, and 
innovation - Convene 
stakeholders to explore 
potential policies to 
improve - Articulate 


Creation and 
dissemination of 
updated policies 


To implement 
appropriate 
processes to 


Policies that meet 
the needs of quality 


Policy environment 
that improves 


Effective 
implementation of 


Policies quality in online and innovative 2 ie : : for quality in Administrators 
: : ‘ : support online updated policies online teaching and ; : 
teaching, learning, | online teaching and : : : : online teaching, 
: : P teaching and toward quality online | learning : 
and innovation learning i : : learning, and 
learning teaching and learning - : E 
innovation 


Ensure full compliance 
with the 
nationally/locally 
approved curricula and 
teaching/learning 
outcomes 


- Review existing 
processes and modify 
for evaluating quality 
of online teaching Teachers delivering 
performance - Identify | quality online 
and develop additional | instruction, with - Short term: 
Processes that To create clear processes needed to both administrators | Clearly articulated 
clarify teacher processes for support teachers - and teachers teacher evaluation | Administrators, 
performance evaluating teacher | Consider feedbackon | understanding the | processes - Long 
evaluation performance the existing policies criteria and quality | term: Teacher 
and introduced standards for evaluation results 
initiatives from all teacher 
stakeholders to consult | performance 
interests of the entire 
education community 


Clarity on processes 
for evaluating 
Process teacher 
performance in 
online teaching 
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1 i ONLINE LEARNING 
Bam Cleontoariun 


- For equitable 
resource allocation: 
Explore current 
resource allocation 
policies and processes 
to identify potential 
gaps between school 
designations; Establish 
plan to remediate any 
identified gaps - For 
incentivizing teachers 


- Equitable resource 
allocation (physical 
devices, teacher 
training) across 
urban and rural 
schools - Incentives 


- Explore 
opportunities for 
equitable resource 
allocation for 
schools - Processes 
that incentivize 


- To create more 
equitable resource 
allocation for 
schools across 
geographies - To 
further incentivize 


- Capacity to deliver 
more equitable 
distribution of 
physical devices 
and resources - 


- Equitable access 
to resources 
across school 
contexts - Teacher | Administrators 


Resource 


Allocation and/or teacher to participate in Teachers’ A a 
; RAT es teachers to ; : participation in 
compensation for | participation in Re ‘ professional learning: | responses to ; 
oe eh participate in : 3 : professional 
teacher training training and Establish core and/or _| incentives for 


professional 
learning for online 
teaching 


and professional learning 


learning related to 
online instruction 


professional 
learning for online 
teaching 


professional 
learning 


essential offerings for 
teacher professional 
learning related to 
online teaching; 
Identify and share 
funding opportunities 
or incentives with 
teachers 


- Establish regular 
opportunities and light 
incentives for 
Expanded drive and To enhance a innovation - Identify School level Number of school- 
incentivization for | Create pathways for | culture of barriers to innovation | stakeholders feel level stakeholders 
Staffing/ innovation onthe | innovation in online | innovation and and “pain points” safe seeking to who show 
Innovation part of teaching and service across and/or “fear factors” innovate without increased 
administrators, learning school level that may prevent reprisal or innovative 
staff, and teachers stakeholders innovative behavior on | punishment behaviors 
the part of 
administrators, staff, 


Administrators 


and teachers 
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- Review the existing 
frameworks used for 
instructional design - 
Identify with 
instructional designers. 


Utilize and 


articulate to school ; Percentage of 
: ; and curriculum 
stakeholders - To disseminate teachers who 
i developers the most F . 
teachers, in Embrace awareness and : Quality online report better 
; . 5 ; tale appropriate : : 
Instructional particular - the instructional design | importance of : teaching and preparation to ae 
3 on : 5 : ; frameworks going : : i Ministry of 
Design Methods | specific practices like UDL | instructional : learning across K- | deliver curricula, : 
forward with strong Education 


& UDL instructional design 
frameworks, 
including UDL, in 
the design of 
curricula 


explicitly in the 
design of curricula 


design 
considerations of 
UDL 


12 schools in the regardless of the 

Kingdom fluidity of face-to- 
face, hybrid, or 
online instruction 


support for UDL - 
Introduce stimuli for 
increasing teacher 
motivation for 
innovative tech-based 
practice - Develop 
means for proactive 
pedagogue retention 


Curriculum 
Design & 
Planning 


Create - Teachers take time at 

opportunities for To guide students | the beginning of 

student to articulate for courses, terms, and 

participation in themselves what | content units to review | Students and 
meaning-making the learning course learning teachers are 
around course objectives mean |objectives-Teachers _| engaged in high 
learning objectives, |andto knowhow |askstudentstocreate | quality online 
and to articulate they can assess shared understandings | teaching and 
what success their own progress | of course learning learning 

means in a way that | toward those objectives and what 

they can learning objectives | success means in 

understand and relation to those 

begin to own 


Help students 
understand and 
make meaningful 
connections to the 
learning outcomes 
and their required 
performance to 
meet those 
objectives 


: Student mastery 
Learning 


Objectives 


of course learning | Teachers 
objectives 
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: Teachers clarif Tocr 
Alignment between faa eeu 6 cals Schoolteacherteams | Regular Percentage of 
learning outcomes | alignment : ae 
content and : : ; hold regular meetings | communications learners that 
: ere that align with between learning : 
learning activities se oatines sorties andthe to peer review from teachers back | report that 
Alignment to the assessments | ® alignment of learning | to the Ministry of instruction, Teachers 


regarding assessments and 


and learning : aN outcomes to Education/NELC assignments, and 
gare eae : standards for learning objectives s é : 

objectives in online |. : ; 5 curriculum and sharing their content meets 

: ; instructional in online : : . . s 

instruction instructional planning | learnings their needs 


planning instruction 


Ensure that course 


materials and To continually 
, ? - Percentage of 
content included - Regularly review _| ensure that course | - Develop regular 
s ; : : : Course content and | course content 
are aligned to course materials materials are review and updating 5 : 
materials are and materials 


assessment and and content - aligned to processes and 

Course Materials | learning objectives, | Regularly update assessment and timelines for course 

& Content are of a high course materials learning content and materials - 
quality, accessible, | and content with objectives, are of a | Seek opportunities to 
and are regularly current and media- | high quality, promote OER in course 
updated with rich sources accessible, and are | materials 
current and media- regularly updated 
rich content 


regularly reviewed | reviews completed 

for currency and - Number of 

resource content resources updated 

and updated as with current and 

appropriate media-rich 
content 


Ministry of 
Education 


- Incorporate 
innovation efforts as a 
metric for performance 
evaluations - Explore 
To promote a possibilities for 
supportive climate | incentivizing or 
Encourage teachers | and culture for promoting innovative | Teachers feel Reported 
to be more teacher innovation | design efforts (e.g. supported and enthusiasm to 
pedagogically or with respect to regular sharing encouraged to new lessons by Administrators 
technologically curricular designs | opportunities at innovate in their students and 
innovative and redesigns, professional learning or | instructional design | teachers 
such as design team meetings; 
thinking celebrating “what 
didn’t work” as a way 


Expand the 
opportunity set for 
teachers to 
innovate with their 
curricular and/or 
instructional design 


Innovation 


to workshop innovative 
lesson improvements; 
solicit submissions for 
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Communication 


Engagement 


Online 
Teaching & 
Learning 


Expectation 
Setting 


Policies and 
processes for 
expectations 
around 
communications 


Increase 
opportunities for 
students to interact 
with their fellow 
students and the 
teacher 


Expectations for 
student activity are 
clearly defined and 
communicated to 
students 


Establish 
expectations for 
responsiveness 


Implement both 
formal and informal 
opportunities for 
students to interact 
with each other and 
the teacher 


Develop, define, 
and communicate 
expectations for 
student activity 


To set 
expectations 
regarding when 
students can 
expect to 
communicate with 
the teacher 


To increase 
student 
engagement with 
course content, 
their peers, and 
the teacher 
through 
academically 
challenging 
components 


To increase 
presence, 
communication, 
and awareness of 
student activity 
expectations to 
support successful 
student learning 


a periodic/regular 
innovation award) 
Develop, define and 
communicate 
expectations for when 
students can expect 
responses (to emails, 
phone calls, etc.) from 
the teacher 


Include intentional 
opportunities for 
academically 
challenging 
interactions with 
students and the 
teacher in course 
design (formal) and 
delivery (informal) 


- Develop and define 
student activity 
expectations - 
Communicate 
expectations to 
students 


Established 
parameters for 
when teachers will 
respond to student 
communications 


Stronger 
connections 
between the 
students and the 
teacher 


- Student activity 
expectations are 
present in the 
syllabus - Student 
activity 
expectations are 
communicated to 
students 


Teacher response 


: Teachers 
time 


- Quality course 
reviews - Presence 
of academically 
challenging 
interactive 
opportunities - 
Percentage of 
students who 
indicate that 
interactive 
opportunities are 
present and 
academically 
challenging - 
Teacher 
satisfaction 


Teachers, 
Students 


- Quality course 
review - 
Student activity 
expectations 


Teachers 
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Oye C} ONLINE LEA 


Outcomes 


Course 
Interaction 
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JIM 


Measure, review, 
and communicate 
student learning 
outcomes in online 
courses, and 
provide 
comparative onsite 
data as appropriate 


Rubrics for all 
students learning 
activities and 
assignments 


Policies and 
processes regarding 
interactions in the 
online classroom 


- Develop and 
implement student 
learning outcome 
metrics for online 
courses - 
Communicate 
findings, including 
onsite comparisons 
as appropriate 


Clearly articulate 
criteria for 
assignments 


Support teacher-to- 
student and 
student-to-student 
interactions 


To develop and 
communicate an 
understanding of 
student learning 
outcomes in 
online courses, 
including 
comparative 
onsite data as 
appropriate, to aid 
in both continual 
course 
improvements as 
well as to provide 
accurate 
perceptions of the 
efficacy of online 
courses 

To provide 
students with 
clearly defined 
criteria for the 
grading of all 
assignments 


To encourage 
teacher-to- 
student and 
student-to- 
student 
interactions in 
online classes 


- Develop student 
learning outcome 
metrics for online 
courses - Measure 
student learning 
outcomes in online 


courses - Communicate 


findings, including 
comparative onsite 


data as appropriate - 
Use findings to aid in 
continual improvement 


of courses 


Develop rubrics linked 


to student learning 
outcomes to show 
grading criteria 


- Develop and 


implement policies and 
process at the national 


and local level to 


encourage interactions 


in online classes - 
Teachers need to 


encourage and ensure 


opportunities for 


- Student learning 
outcome metrics 
are developed and 
implemented for 
online courses - 
Results of these 
metrics and 
comparative data 
are communicated - 
Findings are utilized 
in continual 
improvement 
processes 


Increased 
awareness of 
grading criteria 


- Teacher-to- 
student and 
student-to-student 
interactions occur 
in the online 
classroom 


- Student learning 
outcomes - 
Frequency of 
communication - 
Percentage of 
administrators, 
staff, and teachers 
that indicate that 
they understand 
student learning 
outcomes data for 
online courses - 
Administrator, 
staff, and teacher 
perceptions 
regarding the 
efficacy of online 
courses 


Percentage of 
assignments and 
activities that have 
developed rubrics 


- Student 
satisfaction with 
interactions - 
Teacher 
satisfaction with 
interactions 


Administrators, 
Staff, Tteachers 


Teachers 


NELC, Ministry of 
Education, 
Administrators, 
Staff, Teachers 


Feedback 


Opportunities for 
the student to get 


Establish online 
presence of the 


to know the teacher | educator 


Feedback provision 


and methods, 
timing, and 
connection to 


learning objectives 


is present and 
communicated to 
students 


Develop and 
communicate 
methods for 


providing feedback, 


and set 
expectations for 
timing and 
explanations of 
how feedback is 


related to learning 


objectives 


To provide 
opportunities for 
the student to 
better know their 
teacher 


To support 


successful student 
learning outcomes 


by establishing 
and 
communicating 
clear processes 
and expectation 
for feedback 


interactions in their 
classes 


- Include a teacher 
biography and contact 
information - Share 
expertise with students 
through external 
resources (especially 
those written/prepared 
by the educator, such 
as teacher portfolios) - 
Regularly participate in 
discussions or other 
collaborative activities 


- Develop processes 
and methods of 
feedback provision in 
alignment with course 
activities - 
Communicate 
feedback processes 
and methods, and set 
expectations including 
timing and how 
feedback is connected 
to learning objectives 
and to students 


Increased 
engagement 
between the 
teacher and 
students 


- Feedback 
processes and 
methods are 
developed and 
implemented - 
Communication to 
students regarding 
feedback 
processes, 
methods, 
connection to 
learning objectives, 
and expectations is 
provided 


Frequency of 
teacher 
contributions in 
the virtual 
classroom 


- Percentage of 
teachers who 
implement 
processes and 
methods of 
feedback - 
Frequency of 


communication to 
students regarding 


feedback - 
Percentage of 


students reporting 


that feedback 
processes, 
methods, and 
provision are 
implemented, 
understood, and 
support learning 


Teachers 


Teachers 
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Innovation 


Assessment 
Strategies 


UM 


- Encourage the 
discovery, 
dissemination, and 
implementation of 
innovation - 
Provide 
professional 
development on 
the design, 
development, and 
delivery of 
innovative teaching 
and student 
support 


Provide insightful 
information on 
assessments, their 
objectives and 
overall learning 
outcomes to 
students 


- Explicitly 
encourage 
innovation through 
communications, 
activities, and 
professional 
development - 
Develop and 
implement 
professional 
development 
opportunities on 
innovative teaching 
and student 
support 


Elaborate and 
implement subject- 
related 
communications to 
students regarding 
assessments, their 
objectives and 
overall learning 
outcomes 


To increase the 
design, 
development, and 
delivery of 
innovative 
teaching and 
student support 


To ensure student 
understanding 
and appreciation 
of assessment 
activities to 
increase learning 
efficiency and 
promote 
successful 
educational 
outcomes 


- Develop and 
implement strategies 
for the discovery and 
dissemination of 
innovative teaching 
and student support 
techniques - Develop 
and implement 
professional 
development 
opportunities for 
teachers and staff on 
the design, 
development, and 
implementation of 
innovative teaching 
and student support 


Develop and provide 
communications to 
students via various 
media (syllabi, 
webinars, email, LMS 
communications, fora, 
etc.), clearly identifying 
assessment strategies, 
processes, 
methodologies, types 
of assessments, their 
objectives and overall 
learning outcomes 


- Innovation 
actively 
encouraged - 
Professional 
development 
opportunities 
related to 
innovative teaching 
and student 
support are 
provided 


Schools provide 
insightful and 
accurate 
information about 
the existing and/or 
developed 
assessment types 


and procedures, the 


rationale behind 


these strategies and 


their clearly- 
worded objectives 


- Frequency of 
communications 
to teachers and 
staff encouraging 
innovation - 
Teacher and staff 
participation in 
professional 
development 
opportunities - 
Percentage of 
teachers and staff 
that implement 
and measure 
innovative 
practices 


Administrators, 
Staff, Teachers 


Frequency of 
communication to 
students for 
comprehensive 
and up-to-date 
information on 
ongoing 
assessment 
strategies 
(formative and 
summative 
assessment), their 
objectives, overall 
learning outcomes 
and educational 
impact 


Administrators, 
Teachers, 
Students 


Assessment 
Processes 


Ensure alignment of 
learning objectives 
and assessments 


Align subject 
materials, including 
criteria, 
assignments, 
expectations, and 
grading rubrics with 
best practices 


Promote open 
dialogue with all 
members of the 
educational 
process 


Ensure that 
assessments pursue 
generally accepted 
best practices and 
directly correlate to 
teacher 
expectations 


To support 
successful student 
learning through 
setting out clear 
and direct 
objectives for 
what is expected 
nationally of 
students, 
teachers, schools 
and the entire K- 
12 system where 
assessment serves 
to advance 
teaching and 
learning 


To support 


student excellence 


in learning 
outcomes through 


the use of effective 


course design and 
assessment for a 
more student- 
focused, 
competency- 
based education 


- Align assessment with 
national goals for 
student learning - 
Create constructive 
interaction between 
school administration, 
teachers and students 
to help students 
progress and be even 
better motivated to 
learn and achieve 


- Align subject-related 
assessments with 
adequate course 
design, including 
delivery and placement 
- Ensure that 
assignments, criteria, 
expectations, and 
grading rubrics are 
present and that these 
elements are timely 
and clearly 
communicated to 
students 


Assessments and 
reporting of results 
are aligned with the 
learning standards, 
so that teachers, 
students, school 
administration and 
parents have a clear 
understanding of 
students’ 
advancement 
and/or of any gaps 
and difficulties in 
their learning 


- Subject-related 
assessment delivery 
and placement 
follow the 
established course 
design standards - 
Assignments, 
criteria, 
expectations, and 
grading rubrics are 
present and are 
timely and clearly 
communicated to 
students 


- Student learning 
outcomes met - 
Alignment of 
learning objectives 
with assessments 


Administrators, 
Teachers 


- Quality reviews - 
Number of 
communications 
concerning 
assignments, 
criteria, 
expectations, and 
grading rubrics 
meaningfully 
provided to 
students - Student 
learning outcomes 
met - Course 
evaluations and 
feedback 


Administrators, 
Teachers 
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UM 


Ensure a greater 
variety of 
assessment 
methods to enable 
most accurate 
student subject 
proficiency and 
individual student’s 
mastery of the 
curriculum in 
response to 
instruction and self- 
education 


- Development, 
dissemination, and 
implementation of 
innovative (ICT- 
based) assessment 
strategies, 
processes, 
methods, and types 
- Provide 
professional 
development on 
the design, 
development, and 
delivery of 
innovative 
assessment 


Use diverse 
assessment 
approaches and 
techniques to 
better meet various 
students’ needs 


Encourage 
innovative 
assessment 
through 
communications, 
activities, and 
related professional 
development - 
Develop and 
implement 
professional 
development 
opportunities on 
innovative (ICT- 
based) assessment 


To raise levels of 
student success 
and achieve 
greater equity of 
learning outcomes 
through more 
frequent, 
transparent, 
engaging and 
interactive 
assessments 


To beneficate 
teacher-student 
interaction and 
enhance the ways 
in which students 
receive 
evaluations and 
engage with 
feedback 


Develop, gain approval 
and implement 
engaging assessment 
methods for students 
(peer- and self- 
assessment) 


- Develop and 
implement strategies 
for the discovery and 
dissemination of 
innovative assessment 
techniques - Design 
various ICT-based 
assessment 
components - Conduct 
follow-up on the use of 
innovative assessment 
activities to ensure 
timely improvements 
and avoid confusion - 
Develop and 
implement 
professional 
development 
opportunities for 
teachers on the design, 
development, and 
implementation of 


- Peer- and self- 
assessment 
methods are 
developed and 
used on a regular 
basis - Students 
gain diversified 
opportunities to 
display their 
subject proficiency 
and individual 
mastery of the 
curriculum 


- Innovation 
relevant to 
assessment is 
actively 
encouraged and 
supported - Various 
ICT-based 
assessment 
components 
developed and 
implemented; 
student responses 
obtained - 
Professional 
development 
opportunities 
related to 
innovative 
assessment are 
provided 


- Quality ratings of 
student learning 
outcomes - 
Frequency of 
transparent, 
engaging, and 
interactive 
assessments 


Teachers 


- Number and 
clarity of 
communications - 
Participation in 
professional 
development 
opportunities 
related to 
innovative 
assessment 
methods - 
Percentage of 
teachers who 
implement and 
measure 
innovative 
assessment 
practices in their 
practice 


Administrators, 
Teachers 


Technology 


Centralized 
Online Education 
Infrastructure 
(SSO, LMS, etc.,) 


Operability 


Modality 


Increase access toa 
centralized online 
learning program 


Improve the ability 
for students and 
parents to use 
learning 
technologies 


Understand the 
student experience 
in online learning to 
improve courses 


Identify schools 
that have not yet 
implemented 
online learning and 
providea 
centralized LMS 
with SSO 


- Develop a 
feedback loop to 
ensure parent and 
student needs are 
met - Create 
centralized 
resources that are 
comprehensible 
and policy 
accessible for 
parents and 
students 


- Evaluate student 
experience over 
time - Reflect on 
course design and 
areas for 
improvement 
based on needs 


To increase the 
number of schools 
successfully 
implementing 
online learning 


To improve 
understanding 
about using 
learning 
technologies 


To better support 
students and 
improve learning 
outcomes 


innovative assessment 
methods 


- Identify schools who 
do not have a LMS - 
Help schools determine 
any contextual needs - 
Ensure interoperability 
standards between the 
LMS and other systems 
- Implement SSO 


- Create a mechanism 
for parents and 
students to ask 
questions about 
learning technologies - 
Curate relevant 
resources related to 
challenges - Develop an 
open, centralized space 
for parents and 
students to share 
common pain points 
for the school and 
access resources 


- Collect feedback from 
students about 
asynchronous and 
synchronous learning 
experiences - Evaluate 
feedback and use to 
inform appropriate 
course changes 


- Schools with 
needs identified 
and contextualized 
- LMS and SSO 
adopted and 
implemented - 
Students 
successfully log into 
LMS 


- Parents and 
students share 
challenges - 
Students have 
access to 
technology - 
Curated resources 
are easy to find and 
access 


- Students sharing 
feedback - 
Feedback is 
implemented into 
course design 


Percentage of 
schools with an 
LMS and SSO 


Ministry of 
Education 


- Percentage of 
students who can 
use learning 
technologies - 
Website visits, 
page views, and 
hits 


Students 


- Student 
outcomes - 
Frequency of 
feedback 
collection 


Teachers, 
Students 
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Administrators, 
Staff, Parents, 
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Security 


ITSM Compliance 
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- Increase trust and 
confidence in for 


data security - Clear 


policies and 
practices to ensure 
data security 


Evaluate current 
processes, plan, 
and resources to 
ensure successful 
online course 
delivery 


- Develop a clear 
data security plan 
and share with 
stakeholders - 
Develop policies 
around the use of 
third-party 
applications 


To decrease 
privacy and 
security risks 


- Determine IT pain 
points among 
stakeholders - 
Evaluate current 
processes, plans 
and resources - 
Develop new 
resources to 
address pain points 


To increase 


course delivery 


support for online 


- Educate practitioners 
and families about the 
risks of using third- 
party applications that 
have not been 
appropriately vetted - 
Create a process for 
vetting applications 
and share with 
practitioners - Develop 
guidelines for student 
data use and share 
with stakeholders 


- Communicate with 
different stakeholder 
groups (staff, students, 
teachers, parents) to 
determine pain points 
(survey, focus group, 
town hall, etc.) - Use 
identified pain points 
to examine current 
internal and outward- 
facing resources and 
address gaps or areas 
to improve - Create 
new resources and 
share in a manner that 
is easy to find 


- Practitioners 


avoiding potentially 


malicious 
applications - 
Stakeholders 
participating in 
conversations - 


Sufficient expertise 


to vet technologies 


and use of ministry- 


provided websites 
and applications 


- Various 
stakeholders 
engaged - Pain 
points determined 
and addressed 


- Number of Administrators, 
applications in Staff, Teachers, 
schools - Reported | Parents, 

trust Students 


- Number of 
support 
tickets/requests - 
Number of 
resources 
accessed - 
Stakeholder 
satisfaction 


Administrators, 
ITSM 


Internet Access 


Reliability 


Determine and 
address student 
internet access 
needs 


Identify system- 
level and/or 
infrastructure 
challenges related 
to the reliability of 
access to learning 
technology systems 
and courses 


- Identify student 
internet needs- 
Develop a 
distribution plan to 
share mobile 
hotspots - Evaluate 
course content to 
ensure options for 
students to 
continue work with 
poor internet 


Develop processes 
for monitoring and 
addressing the 
reliability of 
infrastructure and 
learning technology 
systems and 
courses 


To maintain 
continuity of 
learning to the 
greatest possible 
extent 


To increase the 
awareness and 
problem-solving of 
system-level 
infrastructure 
reliability as 
related to access 
of learning 
technology 
systems and 
courses 


- Communicate with 
families through 
different channels to 
determine need - 
Procure and distribute 
mobile hotspots with a 
guide for use - Maintain 
a feedback loop for 
future internet access 
needs - Teachers 
evaluate content to 
minimize the amount 
of synchronous online 
sessions, high 
resolution recordings 
and images, etc.- 
Teachers develop print 
contingency materials 
for students to 
complete in the event 
of a sudden, sustained 
loss of internet 


- Develop processes for 
monitoring reliability - 
Develop processes for 
problem-solving 


system reliability issues 


- Implement solutions 
to address reliability 
challenges 


- All students have 
access to internet 
to complete their 
work - Families can 
report loss of 
internet and receive 
technology after 
start of the school 
year - Teachers 
trained on how to 
vet high-bandwidth 
content 


- Processes for 
monitoring 
reliability are 
developed and 
implemented - 
Processes for 
resolving reliability 
challenges are 
developed and 
implemented 


- Percentage of 
students who have 
access to internet - 
Number of 
hotspots 
distributed - 
Percentage of 
students who 
continue to make 
satisfactory 
academic progress 


Administrators, 
Staff, Teachers, 
Students 


- Number of 
system reliability 
issues - Feedback 
from stakeholders 


Administrators, 
ITSM 
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Coverage 


Innovation 


RNING 
JM 


Developing 
mechanisms to 
determine 
stakeholder needs 
and ways to 
appropriately 
address them 


Increase training on 
educational data 
and learning 
analytics tools and 
approaches in 
educator 
preparation 
programs 


- Stakeholders have 
multiple ways to 
report needs - 
Periodic follow-ups 
and opportunities 
to acquire new 
technologies 


Curriculum 
modification to 
include gaining 
experience working 
with student data 


at different levels of 


granularity 


To decrease 
coverage gaps for 
learning 
technologies 


To increase data 
literacy and 
understanding of 
analytics to 
improve student 
outcomes 


- Determine 
appropriate 
communication 
channels for 
stakeholder groups - 
Communicate via all 
channels and follow-up 
with non-respondents - 
Maintain feedback loop 
to ensure coverage 
throughout the year - 
Ensure that schools 
have enough 
technologies to 
support student 
remote/online learning 


- Work with educator 
preparation programs 
to evaluate and revise 
curriculum for 
developing data 
literacy and analytics 
skills - Develop and 
provide sustained 
training at different 
experience levels for 
practitioners - Provide 
time for teachers to 
work with their 
students’ learning data 


- Stakeholders 
report needs - 
Continued 
communication 
and opportunity to 
determine needs - 
Adequate 
technologies to 
support needs - 
Needs are 
evaluated and 
technologies are 
provided 


- Educator 
preparation 
programs and 
curricula are 
revised to include 
skill development 
over the course of 
the program - 
Training is 
sustained and 
builds skills over 
time instead of one- 
off sessions - 
Teachers have time 
incentive to explore 
data 


- Percentage of 
stakeholders have 
appropriate 
access over time - 
Number of 
technologies to 
meet needs 


- Number of 
educator 
preparation 
programs that 


revise curriculum - 


Number of 


schools - 
Percentage of 
practitioners that 
attend trainings - 
Blocks of time to 
explore data 


Administrators, 
Staff, Teachers, 
Students 


Ministry of 
Education, NELC, 
trainings offered in| Universities, 


School 
Administrators 


Student 
Orientation & 
Support 


Clear and 
comprehensive 
orientation 
provided that 
includes all 
necessary elements 
of technology, class 
design, academic 
support resources, 
and technological 
support resources 


Provide academic 
support resources 
that are accessible 
online 


Develop and 
implement 
comprehensive 
online course 
orientations 


- Develop sufficient 
academic support 
resources - 
Implement digital 
access for academic 
support resources 


To increase 
student awareness 
of and ability to 
access support 
needs for courses 
as well as to 
effectively use 
learning 
technologies used 
in courses 


To increase 
student awareness 
of and ability to 
access academic 
support resources 


Develop and 
implement 
comprehensive online 
course orientations 


- Develop academic 
support resources - 
Implement digital 
access for academic 
support resources 


Orientations are 
developed, include 
all necessary 
elements, and 
provided to 
students 


Academic support 
resources are 
developed and 
accessible to 
students online 


- Percentage of 
students that 
complete online 
course 
orientations - 
Percentage of 
students that 
indicate 


awareness and use 


of support 
reSOuIces - 
Number of 
support tickets 
related to 
students’ ability to 
effectively 
navigate courses 
and use course 
technologies 


- Percentage of 
students that 
indicate 
awareness of 
available 
academic support 
resources - 
Students 
access/engageme 
nt with academic 
support resources 


Administrators, 
Staff, Teachers, 
Students 
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Develop and 
implement 
evaluations to 
assess student 
support needs 


Develop and 
implement 


- Development of 
student support 


To increase the 


Develop and Administrators, 


é ability of school é : : - Evaluation 
related to the appropriate ee : implement appropriate | evaluations - Data el Staff, Teachers, 
ae : to identify student é ; completed - Data 
provision of evaluations to evaluations to assess __| collection from Parents, 
é ; support needs and collected 
equitable education | assess student rouide colutione student support needs | student support Students 
opportunities, support needs P evaluations 


including financial, 
health, and 
academic areas 


- Development of 
To increase the student support - Implemented 
ability of schools services that are student support 
to provide student accessible via services - Number 
support regardless distance - Provision | of times student 
of whether of student support | support services 
students are services that are are accessed by 
onsite or online notreliantonon- | students 

site presence 


Develop and Develop and 
implement student | implement student 
support solutions | support solutions 
that are not reliant | that may be 

on on-site delivery | accessed ata 
methods distance 


Develop and 
implement appropriate 
student support and 
delivery methods 


Administrators, 
Staff, Teachers 


- Percentage of 
courses reviewed 
Review the use of - Courses are for adherence to 
accessibility standards | reviewed for accessibility 
in online courses and | adherence to standards - 
ensure that they are accessibility Percentage of 
duly met/maintained _| standards - Any gaps identified 
to the fullest possible | gapsidentified are | and addressed - 
extent addressed Access and 
participation by 
students with 
disabilities in 


Ensure that online |Reviewtheuseof | Toincrease the 
courses follow accessibility ability of students 
accessibility standardsin online | with disabilities to 
standards to courses and ensure | fully access and 
support students that gaps are participate in 

with disabilities addressed online learning 


Administrators, 


Accessibility Staff, Teachers 


online learning 


168 SSS, v“ 


Compliance 
Standards 


Social Emotional 


Ensure that online 
course accessibility 
is in compliance 
with standards 


Develop and 
implement 
evaluations to 
assess student 
support needs 
related to the 
provision of social 
emotional services 


Develop and 
implement social 
emotional student 
support solutions 
that are not reliant 
on on-site delivery 
methods 


Review online 
courses for 
compliance with 
accessibility 
standards 


Develop and 
implement 
appropriate 
evaluations to 
assess social 
emotional student 
support needs 


Develop and 
implement social 
emotional student 
support solutions 
that may be 
accessed ata 
distance 


To increase the 
compliance of 
online courses 
with accessibility 
standards 


To increase the 
ability of schools 
to identify social 
emotional student 
support needs and 
provide solutions 


To increase the 
ability of schools 
to provide social 
emotional support 
to students 
regardless of 
whether students 
are onsite or 
online 


Review courses for 
compliance with 
accessibility standards 
and address courses 
where compliance is 
not met 


Develop and 
implement appropriate 
evaluations to assess 
social emotional 
student support needs 


Develop and 
implement appropriate 
social emotional 
student support and 
delivery methods 


- Courses are 
reviewed for 
compliance with 
accessibility 
standards - Areas in 
which compliance 
are not met are 
addressed 


- Development of 
social emotional 
student support 
evaluations - Data 
collection from 
student support 
evaluations 


- Development of 
social emotional 
student support 
services that are 
accessible via 
distance - Provision 
of social emotional 
student support 
services that are 
not reliant on on- 
site presence 


- Percentage of 
courses reviewed 
as compliant with 
accessibility 
standards - 
Percentage of 
gaps that are 
identified and 
addressed - 
Percentage of 
compliance 
standards met in 
online courses 


Administrators, 
Staff, Teachers 


Administrators, 
Staff, Teachers, 
Parents, 
Students 


- Evaluations 
completed - Data 
is collected 


- Implemented 
student support 
services related to 
social emotional 
needs - Number of 
times student 
support services 
related to social 
emotional needs 
are accessed by 
students 


Administrators, 
Staff, Teachers 
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Innovation 


Technical 
Assistance 


LEARNING 
jm 


Opportunities to 
develop social 
emotional support 
in normal course 


engagement, 
including 
relationship- and 
community- 
building 


Increased and 
effective 
communication in 
online courses 


Further improve 
technical assistance 
mechanisms for 
educators and 
students during 
online activity 
development and 
online teaching and 
learning 


Develop and 
implement 
opportunities for 
students to build 
relationships and 
communities within 
their courses 


Discover and 
implement 
innovative methods 
of communicating 
with students in 
online courses 


-Createa 
centralized support 
space througha 
website/education 
portal - Resources 
explicitly shared 
and stressed by 
teachers in syllabi 
and online course, 
and school 


To increase social 
emotional support 
of students 
through 
opportunities for 
relationship- and 
community- 
building 


To improve 
communication 
methods in online 
courses 


To remove 
technical 
obstacles for 
teachers’ 
pedagogical 
success and 
students’ learning 
progress - Develop 
and implement 
online resources 
for common use 
and contribution 


Develop and provide 
opportunities for 
students to build 
relationships and 
communities in the 
context of their regular 
course engagement 


Discover and 
implement innovative 
methods of 
communicating with 
students in online 
courses 

- Create expert teams 
of stakeholders at 
schools including 
student members to 
inspire and guide the 
development of most 
required resources - 
Create a centralized 
portal/website with 


curated resources (e.g., 


guides, videos, online 
video lectures, 
frequently asked 


questions) and support 


mechanisms (e.g.: live 
chats, chatbots, fora) - 
Communicate 
resources through 
Future Gate, online 
courses, and syllabi 


Opportunities are 
present for 
students to build 
relationships and 
communities within 
their online 
courses 


Innovative 
communication 
methods are 
implemented in 
online courses 


- Stakeholder 
expertise, adequate 
response to student 
needs and teacher 
requirements - 
Centralized website 
available with 
curated insightful 
resources and 
support 
mechanisms, and 
updated to ensure 
actuality of 
materials and 
urgency of 
assistance provided 
- Resources shared 
through Future 
Gate or alternative 


- Students' social 
emotional support 
opportunities 
within their online 
courses - Student 
participation in 
such opportunities 
to effectively build 
social emotional 
support systems 
Students 
recognize the 
presence and 
effectiveness of 
communication in 
online courses 


Teachers, 
Students 


Administrators, 
Staff, Teachers 


- Page views and 
hits on website - 
Future student 
technical issues 
duly collected and 
considered to 
update the 
resource base - 
Common support 
language in syllabi 
and online courses 
- Shared 
pedagogical 
elaborations and 
successful 
resources 


Administrators, 
Staff, Teachers, 
Students 


education 
platforms, online 
courses, and syllabi 


- Conduct needs 

assessments using data 

from staff surveys to 

identify areas of 
To organize non- | professional Sufficiently and 
stop collaborative | development most adequately 
and job- requested and prepared educators 
embedded anticipated by teaching | to teach online 
professional staff - Identify mentors | through accessible | Teacher efficiency | Administrators, 
developmentto |and coaches (either in- | and adequate and self-efficacy 
increase efficiency | house or through collaborative and 
and and self- expert providers) to job-embedded 
efficacy of guide and support professional 
teachers teacher learning development 

specific to online 

education - Ensure 

access to demo online 

lessons, assignment co- 

designing, co-planning, 

curricular resources, 

including lesson units 

and assessment tools 


Continuing 
professional 
subject- and 
technology-related 
Regular accessto | training and 


comprehensive and | support for 
up-to-date teachers to develop 
professional the knowledge, 
development skills, and abilities 
essential for 


Professional 
Development 


designing and 
giving online 
classes 
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Increase 
opportunity for 
teachers to 
complete 
professional 
development and 
remove barriers to 
supporting online 
resources and 
instructions 


Extend and 
reinforce teachers’ 
professional 
advancement and 
promote educators’ 
capacity building 


To provide 
teachers with 
adequate time to 
learn, practice, 


implement, and 
reflect upon new 
strategies that 
facilitate changes 
in their 
professional 
practice 


- Evaluate the use of 
time and redesign 
school schedules to 
increase opportunities 
for teachers’ 
professional 
development, self- 
education and co- 
working - Organize 
active learning, aligned 
with national policies 
and school curricula, 
and provide sufficient 
learning time for 
teachers to engage and 
progress - Ensure 
availability of 
meaningful training 
sessions of convenient 
structure and duration 
within an academic 
year - Break longer 
training sessions into 
more manageable 
elements that can be 
completed over a 
longer period of time 


- Modular content 
that can be 
completed in less 
time - Technical 
support structure 
for short courses - 
Expert group of 
mentoring teachers 
and staff to ensure 
feedback on 
curated resources - 
Teachers made 
aware of resources 
through a variety of 
channels (e.g., 
school website, 
Future Gate or 
alternative online 
platforms, emails, 
etc.) 


- Time teachers 
have to learn, 
practice, 
implement, and 
reflect upon new 
teaching 
strategies - 
Number of 
opportunities to 
engage in learning 
around a single set 
of concepts or 
activities within an 
academic year - 
Students learning 
outcomes as a 
result of efficient 
teacher 
professional 
development 
schemes 


Administrators, 
Staff 


- Generate shared 
vision on high-quality 
teaching standards, 
instruction and 
evaluation - Identify - Education-critical 
widespread technologies 
technologies and identified - 
determine most critical | Regularly updated 
To decrease for development - orientation 
resistance and Partner with relevant | modules to include 
Develop, testand | challenges among | educational new technologies - 
share common teachers for online | institutions to develop | Piloted at schools 
orientation learning and pilot orientation to provide 
modules for schools | technologies and | modules - Use best reasonable and 
improve student | national and actual feedback - 
outcomes international practices | Orientations easily 
of teacher professional | accessible and 
development, which shared with key 
have raised student personnel at 
performance, for better | institutions 
understanding of 
effective and rational 
orientation and 
support 


- Number of 

modules 

developed and Ministry of 
updated Education, 
periodically - Education 
Number of Departments, 
modules accessed | NELC 

and implemented 

by institutions 


Improve access to 
Orientation orientations for 
(Student & widespread and 
Parent) promising 

technologies 


- Teacher needs and 
preferences - Number of 
Design a virtual flexible, : identified - Mentors | mentors and 

eer support (peer | continuing and Ered SST mamee ts metas and coaches mentees - 
Hee = ee Count sesttshsel etn approved and Academic output 

: P : mentees based on their PP : B 

evaluation) and mentoring, assigned - Schemes | (e.g., publications, 
mentoring program | coaching and <procedures and | grants, etc.) - Staff 


To provide - Communicate with 
teachers to identify 
Develop, sustain 
and improve 
opportunities for 
peer mentoring 


Administrators, 


Mentori 
entoring Staff, Teachers 


needs and preferences 


: ; - Provide r rces an Sete ; 
for collegial sustained es g Esoulcrsand criteria duly retention - 

f : incentives where : ; 
learning groups engagement in communicated to _| Teachers’ health 


possible and 


collaboration : 
appropriate 


teachers-mentees - | and wellbeing 
Feedback collected 


and considered to 
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Innovation 


Greater awareness 
and more impactful 
integration of 
advanced 
technologies 


Develop a teacher 
learning 
community 
program 


To build capacity 
around online 
learning tools and 
innovative 
pedagogical 
approaches 


- Teachers forma 
community (or multiple 
communities) - 
Educational 
institutions recruit 
teachers and 
pedagogues into 
communities and 
provide centralized 
coordination 


improve program 
where possible 


- Community 
groups formulate 
their common and 
accepted 
objectives, set 
regular meetings, 
share resources, 
best practices and 
ideas - Clear and 
well-communicated 
expectations for 
program, regular 
meetings, a focal 
point to coordinate 
the group and 
ensure all needs 
and requirements 
are met, incentives 
for participation 
(resources, awards, 
grants, recognition, 
etc.), recruitment 
from all 
departments to 
grow best 
performers 


- Number of 
groups - 
Participation in 
groups - 
Dissemination of 
knowledge and 
skills 


Administrators, 
Staff, Teachers 


- Percentage of 
course and 
programs in 
alignment with 
established 
processes, 
procedures, and 
policies - 
Frequency of 
review and 


To maintain 
and/or increase a 
high level of 
adherence to 
processes, 
procedures, and 
policies for course 
and program 
evaluation, and to 
continually 


- Course and 
program evaluation 
processes, 
procedures, and 
policies are in place 
- Course and 
program evaluation 
processes, 
procedures, and 


Develop and 
implement clear 
processes, 
procedures, and 
policies on course 
and program 
evaluation, 


- Develop and 
implement course and 
program evaluation 
processes, procedures 
and policies - Regularly 
review and update 
course and program 
evaluation processes, 


Processes, 
procedures, and 
policies on course 
and program 
evaluation, 
including review 
and updating, are 


Evaluation of 
Course Outcomes 
and Program 


Quality 


Teachers, 
Parents, 
Students 


Evaluation & 
Continuous 
Improvement 


Student 
Satisfaction 


including regular 
review and 
updating 


Communications to 
students and 
parents regarding 
evaluation 
opportunities, their 
purpose, and how 
feedback is 
incorporated into 
course and 
program 
improvements 


Increased 
communication 


developed and 
implemented 


Create a 
communication 
plan for students 
and parents 
regarding 
evaluation 
opportunities, 
purposes, and uses 


Show students and 
parents how their 
feedback has 
helped to improve 
the online learning 
courses or program 


improve these 
items through 
regular review and 
updating 


To increase 
awareness and 
opportunities for 
student and 
parent evaluations 


To provide clear 
and consistent 
communication to 
students and 
parents regarding 
how their 
feedback has 
improved courses 
and programs 


procedures, and 
policies 


Develop and 
implement a 


communication plan 
regarding evaluation 


opportunities, 


purposes, and uses for 
students and parents 


Develop and 
implement a 


communication plan 


regarding how 
feedback has been 
incorporated 


policies are 
regularly reviewed 
and updated as 
appropriate 


Timely and 
comprehensive 
communications 
are provided to 
students and 
parents regarding 
evaluation 
opportunities, 
purposes, and uses 


Timely and 
comprehensive 
communications 
are provided to 
students and 
parents 


updates for course 
and program 
processes, 
procedures, and 
policies 


Student and 
parent 
participation in 
evaluation 
opportunities 


Parents, 
Students 


Student and 

parent satisfaction | Parents, 
with online 

learning 
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Teacher 
Satisfaction 


Staff Satisfaction 


Innovation 


Communications to 


teachers regarding 
evaluation 


opportunities, their 


purpose, and how 
feedback is 
incorporated into 
course and 
program 
improvements 


Communications to 


staff regarding 
evaluation 


opportunities, their 


purpose, and how 
feedback is 
incorporated into 
course and 
program 
improvements 


Access to research 
and international 
experts 


Create a 
communication 
plan for teachers 
regarding 
evaluation 
opportunities, 
purposes, and uses 


Create a 
communication 
plan for staff 
regarding 
evaluation 
opportunities, 
purposes, and uses 


Implement best 
practices identified 
by online learning 
experts 


To increase 
awareness and 
opportunities for 
teacher 
evaluations 


To increase 
awareness and 
opportunities for 
staff evaluations 


To ensure quality 
of online learning 
by benchmarking 
against best 
practices 


Develop and 
implement a 
communication plan 
regarding evaluation 
opportunities, 
purposes, and uses for 
teachers 


Develop and 
implement a 
communication plan 
regarding evaluation 
opportunities, 
purposes, and uses for 
staff 


- With due regard for 
national and local 
specifics, review quality 
standards (rubrics, 
scorecards, research, 
etc.) shared by online 
learning experts, 
groups, and/or 
organizations - Apply 
best practices to online 
learning courses and 
programs 


Timely and 
comprehensive 
communications 
are provided to 
teachers regarding 
evaluation 
opportunities, 
purposes, and uses 


Timely and 
comprehensive 
communications 
are provided to 
staff regarding 
evaluation 
opportunities, 
purposes, and uses 


Improved quality of 


online learning 


Teacher 
participation in 
evaluation 
opportunities 


Staff participation 
in evaluation 
opportunities 


Staff 
- Use of rubrics or 
scorecards to 


evaluate quality of | Administrators, 
online learning- —_| Staff, Teachers 
Adoption of best 

practices 
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Introduction 


Saudi Arabia's K-12 and higher education sectors had considerable experience in implementing 
online learning before the pandemic. It helped the Kingdom to swiftly move online during the 
pandemic. 


At the national level, the Ministry of Education (MoE) had several initiatives for online learning and 
several solutions targeting all education sectors including public, private, K-12, and the higher 
education sectors. 


To name a few, the launching of the virtual school web portal (https://vschool.sa) used to login all 
K-12 students across the country. This platform includes a number of eLearning tools and a huge 
library of Learning Resources covering almost all curriculums, one of which is len (ien.edu.sa). 
On the other hand, to cater for those who do not have access to the internet or computers, there 
included the launching of 20 live TV channels that broadcast live lessons targeting all levels. 


In the higher education sector, most of the Universities had exclusive Deanship of eLearning who 
in collaboration with the National eLearning Center (NELC) and MoE immediately implemented 
the education continuity plan and moved online with their existing eLearning infrastructure. 


The National OER platform SHMS (shms.sa) with about 0.38 Million resources majority of them 
being in Arabic was readily available for both K-12 and higher education sectors to be used in 
their online teaching and learning. 


In summary, Saudi Arabia promptly responded to the pandemic by an education continuity plan, 
online platform, e-Content, open educational resources, video lessons through YouTube 
channels, and satellite TV for K-12. Similarly higher education institutions had most of these 
solutions delivered independently to their learners. 


Benchmarking 


With this background, a benchmarking analysis was done comparing the national response by 
countries across the world in comparison to the solutions offered by Saudi Arabia for K-12. 


Methodology 


All reference data for the analysis were collected by data mining the information available in the 
public domain. The primary references for the data collection were UNESCO’s National 
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responses and school closures tracker for COVID-19, the World Bank’s report on How countries 
are using edtech (including online learning, radio, television, texting) to support access to remote 
learning during the COVID-19 pandemic and School, OECD Education responses to COVID-19: 
Embracing digital learning and online collaboration, and ISC Research’s Coronavirus COVID-19 
update. In addition, each country's official reports were referenced including the online news 
media. The data collection period was from March-May 2020. 


The indicators for the analysis are listed below: 


Education Continuity Plan (ECP): Presence of education continuity plan 

Post-Covid -Strategy: Presence of plan for reopening the schools at the National level 
Online Platform: A national platform to deliver online teaching and learning 
MOOCS/OER: Availability of OER and/or MOOC at the National Level 

e-Content: Exclusive e-content for online teaching and learning 

YouTube: Exclusive YoutTube channels for video lessons and educational resources 
Satellite TV: Broadcasting lessons through satellite channels. 


ae Or he 


Summary of Analysis: National response by the Government at the country level in the 
percentage of 193 countries analyzed. 


National Response Education Continuity Plan Reopening Strategy National Platform 
87 44 34 53 
% % % % 
i inui A plan for reopening the Fnalionel platen 1 
Response at the An education continuity oe UW ie eiceal deliver online teaching 
National Level plan and learning 
level 
OER e-Content YouTube Satellite TV 
03 
% 41 23 - 
% % " 
Availability of open Exclusive e-content for Exclusive YouTube Broadcasting lessons 
educational resources online teaching and channels for video through satellite 
learning lessons and educational channels 


resources 
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Appendix VI: Recommended Resources 


The following resources (many free) are being provided to support the recommendations provided 


throughout this review. These resources are available through the contributors of this report. 


Dimension Resource Contributor 
Leadership Access and Accessibility in Online Learning: Issues in Online Learning 
Higher Education and K-12 Contexts Consortium 
Emergency Preparedness and Continuity Planning for Online Learning 
K-12 Educators Consortium 
UNESCO Resources UNESCO IITE 
Administrator Guide to Online Learning . Michigan Virtual Learning 
Research Institute 
School Board Guide to Online Learning Michigan Virtual Learning 
Research Institute 
Learning Continuity: Planning Considerations for Michigan Virtual Learning 
School Leaders Research Institute 
ISTE Standards International Society for 
Technology in Education 
(ISTE) 
ISTE Standards for Education Leaders International Society for 
Technology in Education 
(ISTE) 
_ Emergency Remote Instruction Checklist Quality Matters 
| K-12 Communicating New Policies for Online Courses Quality Matters 
National Standards for Quality Online Programs Quality Matters and the 
Virtual Learning Leadership 
Alliance 
Curriculum Guidance on Open Educational Practices during UNESCO IITE 
Design & School Closures 
Planning 


Guidance for Principals and Administrators 


UNESCO IITE and partners 


Handbook on Facilitating Flexible Learning during 
Educational Disruption 


UNESCO IITE and partners 


Guidance on Open Educational Practices During 
COVID-19 Pandemic 


UNESCO IITE and partners 


OLC OSCOR Course Design Review 


Online Learning 
Consortium 


OLC Instructional Designer Programs - Master Series, 


Certificate, and Courses 


Online Learning 


Consortium 
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Launch into Learning Sciences: How Learning Works 


International Society for 
Technology in Education 
(ISTE) 


QM Rubric for Course Review 


Quality Matters 


Applying the QM K-12 Rubric Workshop 


Quality Matters 


K-12 Online Course Design 


Quality Matters 


K-12 Instructional Materials: Investigating What Works 


Quality Matters 


Objectives and Alignment: The Framework for Student 
Success 


Quality Matters 


K-12 Bridge to Quality Design Guide 


Quality Matters 


Online 
Teaching & 
Learning 


Delivering High Quality Instruction Online in Response 
to COVID-19 


Online Learning 
Consortium, 

Every Learner Everywhere, 
Association of Public & 
Land-Grant Universities 


Quality Course Teaching & Instructional Practice 
scorecard 


Online Learning 
Consortium 


OLC Online Teaching & Advanced Online Teaching 
Certificates 


Online Learning 
Consortium 


OLC New to Online Workshops 


Online Learning 
Consortium 


Learning Keeps Going 


International Society for 
Technology in Education 
(ISTE) 


Online Teaching Microcourse 


International Society for 
Technology in Education 
(ISTE) 


Guidance for Teachers 


UNESCO IITE and partners 


Guidance on Active Learning at Home During 
Educational Disruption: Promoting Student’s Self- 
Regulation Skills during COVID-19 Outbreak 


UNESCO IITE and partners 


Handbook on Facilitating Flexible Learning during 
Educational Disruption 


UNESCO IITE and partners 


Teaching in a Digital Age: A 12-Part Video Series on 
Digital Education 


UNESCO IITE and partners 


K-12 Introduction to Teaching Online 


Quality Matters 


Teaching Online Certificate 


Quality Matters 


182 


How the Online Classroom is Different 


Quality Matters 


Learning is not a Spectator Sport: Make it Active! 


Quality Matters 


National Standards for Quality Online Teaching 


Quality Matters and Virtual 
Learning Leadership 
Alliance 


Assessment Creating Effective Assessments Workshop Online Learning 
Consortium 
Assessing Learning in Online Environments International Society for 
Microcourse Technology in Education 
(ISTE) 
Connecting Learning Objectives & Assessments Quality Matters 
Technology ISTE Essential Conditions International Society for 
Technology in Education 
(ISTE) 
Guidance for Using Technologies and Platforms UNESCO IITE and partners 
Personal Data Security Technical Guide for Online UNESCO IITE and partners 
_ Education Platforms 
Personal Data and Privacy Protection in Online UNESCO IITE and partners 
Learning: Guidance for Students, Teachers and 
Parents 
Flip the Switch: Motivate Learners with Course Tools Quality Matters 
and Tech 
QM Gauging Your Technology Skills Workshop Quality Matters 
Student Addressing the Social-Emotional Needs of Remote K- = Online Learning 
Support 12 Learners Webinar Consortium 
Supporting Your Child with Online Learning: A Online Learning 
Webinar for Parents Webinar Consortium 
Student Guide to Online Learning Michigan Virtual Learning 
Research Institute 
Parent Guide to Online Learning Michigan Virtual Learning 
Research Institute 
Guidance for Students UNESCO IITE and partners 
Guidance for Parents and Communities UNESCO IITE and partners 
_ Setting Expectations and Learner Support: Helping Quality Matters 
Learners Be Successful 
Training & OLC Institute for Professional Development Online Learning 
Support Consortium 


ISTE U 


International Society for 
Technology in Education 
(ISTE) 
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_ ISTE Standards for Educators 
| 


International Society for 
Technology in Education 
(ISTE) 


QM Professional Development 


Quality Matters 


Teacher Guide to Online Learning 


Michigan Virtual Learning 
Research Institute 


_ Mentor Guide to Online Learning 


| Understanding the Impact of OER: Achievements and 


Challenges 
Evaluation & OLC Quality Scorecard for the Administration of 


Continuous Online Learning 


Michigan Virtual Learning 
Research Institute 


_ UNESCO IITE and partners 


Online Learning 
Consortium 


Improvement 


-QMK-12 Rubric 


Quality Matters 


QM K-12 Reviewer Course 


Quality Matters 


National Standards for Quality Online Learning 


Quality Matters and Virtual 
Learning Leadership 


| Alliance 


Other Recommended Resources: 


e (Leadership) Digital Learning Collaborative. (2019). Planning for quality: A guide for starting and 
growing a digital learning program. A Keeping Pace with Digital Learning Report. Retrieved from 
https://www.digitallearningcollab.com 

e (Leadership). Consortium for School Networking. (2018). Framework of Essential Skills of the K-12 
CTO. Retrieved from https://www.cosn.org/Framework 

e (Leadership) Google for Education Transformation Center. (n.d.). Retrieved from 
https://edutransformationcenter.withgoogle.com/#/explore 

e (Leadership) Microsoft K-12 Education Transformation Framework. (2020). Retrieved from 
https://www.microsoft.com/en-us/education/school-leaders/k-12-microsoft-education- 
transformation-framework 

e (General) California Department of Education. (2020). Distance learning: Guidance and resources 
for teachers and families in K-12 schools regarding high quality distance learning. Retrieved from 
https://www.cde.ca.gov/ci/cr/dl/ 

e (General) Kennedy, K. (2018). Handbook of Research on K-12 Online and Blended Learning (2nd 
ed.). Carnegie Mellon University. https://doi.org/10.1184/R1/6686813.v1 

e (General) lowa Department of Education. (2020). General resources for educators moving to 
online learning for students during COVID-19. Retrieved from https://educateiowa.gov/general- 
resources-educators-moving-online-learning-students-during-covid-19 
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Appendix VII: Literature Review 


Literature Review 


Introduction 


Online education has rapidly increased worldwide in the past two decades. The implementation of 
online education in K-12 schools began in 1991, and by 1998, many traditional K-12 institutions 
regularly documented the use of online tools and several fully virtual schools had been implemented 
(Barbour, 2019). There are many iterations of online education for K-12 audiences, including the most 
common use of online education for supplementary course support, as well as fully virtual schools. The 
global pandemic environment resulting from COVID-19 has created pressure for rapid development of 
additional quality online education options in the K-12 sector. These needs have converged with a 
period of prolific technology advancements, which have increased opportunities for online education 
and resolved many of the early challenges to teaching and learning in an online environment. 


Online in Saudi Arabia 


General education in Saudi Arabia includes 45 departments of education overseeing the five main 
geographical regions. Despite decreases in educational budgets, general education remains a priority 
for the Kingdom of Saudi Arabia (KSA), with existing initiatives to prepare students for the knowledge 
economy, increase the overall level of the population, and invest in technology and infrastructure 
(Allahmorad, S., & Zreik, 2020). In 2018 alone, 718 new schools were built. Kamal (2009) indicates that 
an explicit focus on the implementation of educational technology has been present in various 
kingdom-level strategic initiatives since 2002. Alahmari and Kyei-Blankson (2016) report that this focus 
has rapidly advanced since 2011. 


The effective development and implementation of digital education infrastructure in K-12 schools is 
essential to the achievement of Vision 2030, and in 2017, the KSA launched the Future Gate initiative, 
designed to provide a country-wide centralized Learning Management System (LMS) reaching 25,000 
middle and secondary schools, 4,500,000 students, and 500,000 teachers (Ohail, Suhaibani, Palavitsinis, 
& Koutoumanos, 2018). The Future Gate initiative includes both technical and practical implementation 
plans, and key performance indicators and usage statistics are measured, with preliminary findings 
indicating broad success across measures. Although there are clear markers of success to date, there 
are several challenges of note that should be considered in the context of pressure for rapid expansion 
of online education amid the pandemic. Infrastructure challenges are focused on technology and 
include the availability and reliability of sufficient internet access, the provision of equipment such as 
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smartboards, projectors, and teacher laptops, and technology for students as not all students are able 
to obtain and/or access a computer. Region- and school-level challenges include orientation and 
training for teachers on the LMS, ongoing and present support for teachers, including the development 
and provision of training and support materials, the development and implementation of quality 
assurance measures and procedures, and change management and motivation. 


The Saudi Ministry of Education recognized these challenges early on, and has taken clear steps to 
develop solutions, primarily based in online education and technology. In 2005, the National eLearning 
Center was established to lead research, development, and implementation of online infrastructure and 
education (NELC, n.d.). In addition, the Ministry of Education has consistently supported the exploration 
and development of online education opportunities through workshops and research (Hamdan, 2014). 


Terminology 


Before reviewing literature on pertinent topics and issues related to online education, a brief discussion 
of terminology is needed. Worldwide, there are a number of terms used to discuss and describe online 
education, and while they are often used interchangeably, there are some differences. Understanding 
those differences and using the same terms and definitions across institutions and environments can 
help ensure that development, implementation, and measurement of initiatives are as effective as 
possible. Singh and Thurman (2019) found that between 1988 and 2018, 46 different definitions of 18 
common terms were used to describe concepts of online learning, sometimes with contradictory 
implications. Further increasing the need to understand and use standard terms and definitions, Moore, 
Dickson-Deane, and Galyen (2011) found that though terms may be used interchangeably, they often 
include different expectations and perceptions. 


While there is still not broad consensus regarding the selection and use of specific terms in the field, 
there are several key considerations of meaning and perception that can guide those seeking to 
establish standard terms and meaning (Singh & Thurman, 2019). The first and most impactful 
consideration for terms is the use of technology. The wide advent of online capabilities in 1995 marked 
the largest shift in terminology. For example, prior to inclusion of the internet, terms such as distance 
learning or were most common, and after, new terms, such as e-learning and online learning emerged. 
Rather than replacing former terms, they added to the lexicon, and are still frequently used 
interchangeably. However, the terms convey information and expectations to teachers and students, 
and selecting and consistently using the most appropriate descriptor can be helpful. Additionally, the 
use and expectations of time should be considered. Time plays an important role in the perception of 
terms, as it can indicate synchronicity or asynchronicity, as well as convey expectations regarding 
meetings and interactions (i.e., online, on-campus, blended), and communicate information about the 
course expectations (for example, in a flipped classroom model, expectations are independent learning 
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of material but interactive participation). In the continued vein of expectation and perception setting, 
the educational context indicated by various terms should be considered. If the institution offers a 
variety of course contexts, such as for-credit, non-credit, continuing education, and/or professional 
development, using clear and distinct terminology for these differences can help teachers and staff 
develop these different courses appropriately, and can help students select and participate per their 
needs. Finally, interactivity is a key delineator of terminology. The key difference is whether the 
distance interface (internet, video, etc.) is used to provide materials and content, or whether there is a 
virtual learning environment in which all course and learning activities take place. Although there is not 
a broad community consensus about specific terminology across the field, these considerations can 
assist those seeking to develop common terms across institutions, systems, or larger environments in 
choosing and implementing the terms most appropriate for their contexts. 


Compounding these challenges and accelerating the need for scalable, effective online education 
opportunities is the current global Coronavirus pandemic. The pandemic forced educational 
institutions across the world to rapidly shift to remote teaching in the middle of the Spring 2020 term, a 
phenomenon that has now given rise to the need to develop and implement long-term solutions much 
more quickly than many had previously planned. 


Online vs Emergency Remote 
Teaching 


Despite the marked increase in courses delivered through the online modality due to Coronavirus - 
UNESCO estimates that 1.5 billion total students (more than 90% of the student population) were 
shifted to fully online courses - it is important to note that there is a difference between this situation, 
referred to as Emergency Remote Teaching (ERT) and online education (Affouneh, Salha, & Khlaif, 2020; 
Bozkurt & Sharma, 2020; Hodges, Moore, Lockee, Trust, & Bond, 2020; Karalis, 2020; Schlesselman, 
2020). Therefore, it is also necessary to understand the differences and how ERT may translate into 
long-term solutions for scalable online education. 


Hodges, Moore, Lockee, Trust, & Bond (2020) succinctly sum up the primary difference between ERT and 
online learning: “In contrast to experiences that are planned from the beginning and designed to be 
online, emergency remote teaching (ERT) is a temporary shift of instructional delivery to an alternate 
delivery mode due to crisis circumstances.” Schlesselman (2020) elaborates on the meaning of online 
learning, stating that “In contrast to emergency remote teaching, online learning is bound by many 
standards to ensure optimal learning. Online learning provides opportunities for community building 
among students and with the instructor; online learning provides opportunities for learner engagement 
with the material through realistic practice, spaced repetition, real-world context, and feedback; online 
learning provides opportunities to support learners. Online learning focuses on the achievement of 
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learning objectives, rather than covering content. Online learning also creates a safe environment that 
addresses the needs of all learners, including accessibility.” 


The assumption with ERT is that it is a stopgap designed to provide as much academic continuity as 
possible during an emergent situation, and that courses will return to their previous modalities 
following the crisis. Coronavirus presents a unique challenge for two reasons. First, as Bozkurt and 
Sharma (2020) and Karalis (2020) point out, this is the first crisis of such scale and severity to occur in 
the current digital knowledge economy. Second, the full impact and trajectory of the coronavirus 
pandemic remains to be seen; educational institutions find themselves in a situation of uncertainty, 
grappling with simultaneous demands to not only provide flexible solutions for the upcoming Fall 2020 
term, but also to design, implement, and scale online education options that were either previously 
unplanned or were not planned with such immediacy (Hodges, Moore, Lockee, Trust, & Bond, 2020; 
Schlesselman, 2020). 


As institutions prepare to make the second large-scale shift resulting from the pandemic, from ERT to 
the design and delivery of online courses, there are several key principles to consider: planning, design, 
and development, pedagogy, quality, communication, technology, student support, and teacher 
support. 


Planning, Design, and Development 


A primary difference between ERT and online learning is the planning and design of the course. For 
online courses, this phase typically ranges from six to nine months or more (though this may be less for 
K-12 given Ministry oversight and provision of curriculum, and provides a foundation for both teachers 
and students to have a successful learning experience (Hodges, Moore, Lockee, Trust, & Bond, 2020; 
Schlesselman, 2020; Shisley, 2020). This phase is a marked difference in which a course is thoughtfully 
created with the goal of effective online learning as opposed to the hurried translation of large 
quantities of material originally prepared for on-campus instruction and improvised solutions that 
accommodate the unexpected change in modality. 


Effective online course design begins with a focus on learning; alignment between learning objectives, 
instructional activities, and assessments, as well as planned immersive, adaptive, and other innovative 
learning experiences. Studies have shown that misalignment among these items is the primary cause of 
poor outcomes. In an emergent situation, attending to these elements is key: reviewing learning 
objectives and revising as appropriate, communicating changes, and providing aligned materials and 
assessments is the most important step toward ensuring learning continuity (O’Keefe, Rafferty, Gunder, 
& Vignare, 2020). 
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It is also critical to consider accessibility when designing an online course, including both web 
accessibility and the broader principles included in Universal Design for Learning (UDL), which shifts the 
focus from accomodations to accessible design for all (CAST, n.d.; O'Keefe, Rafferty, Gunder, & Vignare, 
2020). UDL is beneficial for both planned online learning as well as ERT (whenever possible). For 
example, providing captioned videos ensures accessibility for deaf or hard-of-hearing students, but also 
provides solutions for students who may be non-native speakers, or working in a loud or distracting 
environment. Lastly, teachers should attend to providing not just an accessible environment, but an 
equitable one. In course design, equity may be addressed through review for, and replacement of, 
biased language, images, videos, or other materials. Additionally, online course design should include 
explicit attention to equitable design, including academic, pedagogical, psychological, social, and 
technological considerations (Kelly, 2019), and these elements are included in their respective sections 
below. 


Pedagogy 


While the elements of effective pedagogy are similar in on-campus and online environments, the 
delivery, method, and implementation of instructional strategies is markedly different. At a basic level, 
effective pedagogy builds on course design and aligned objectives, activities, and assessments to 
provide regular and high-quality interaction between students and content, students and peers, and 
students and instructors. In addition to interaction, the Community of Inquiry Framework identifies the 
need for instructors to provide social presence (participant identification with the community), 
teaching presence (design and facilitation of meaningful and educationally valuable learning 
outcomes), and cognitive presence (the ability to construct and confirm meaning) (COl, n.d.). 


In an online course, these elements are carefully designed to align with the learning objectives and 
facilitate mastery of the material. However, this type of immersive experience requires planning and 
preparation, and in an ERT environment, it is most useful to focus on translating on-campus strategies 
that lend themselves well to online alternatives, such as group discussions, writing activities, and 
project-based learning (O’Keefe, Rafferty, Gunder, & Vignare, 2020). 


When considering accessibility and pedagogy, UDL is once again a primary tool in ensuring that optimal 
standards are met. By designing learning for all, it is not just the online experience, but interactions that 
impact students. For example, by providing learning opportunities through multiple means 
(slides/notes, video, a variety of activities), all students may access and learn the same material as 
appropriate for their needs (CAST, n.d.; Kelly, 2019). In addition to following best practices for 
interaction and presence, equity in pedagogy may be developed through the intentional creation of an 
inclusive environment, including authentic and equity-focused strategies for teaching and interaction. 


Quality 


189 


Ol C ONLINE LEARNING 
CONSORTIUM 


Ensuring and maintaining course quality is critical to continued effectiveness in online design, teaching, 
and learning. Course design is foundational for a high-quality course, and teachers may not have the 
knowledge or skills in pedagogy and instructional design to translate their subject-matter expertise 
and/or classroom experience into quality online learning (McGahan, Jackson, & Premer, 2015). There 
are many models which address this potential gap and provide options for evaluation and continuous 
improvement, including the Online Learning Consortium (OLC) Quality Scorecard Suite, Quality Matters 
(QM), and Quality Learning and Teaching (QLT), among others (McGahan, Jackson, & Premer, 2015; 
O’Keefe, Rafferty, Gunder, & Vignare, 2020). 


Though each approach has variations, the goal is to both ensure that key quality indicators are present 
in the course, and to build confidence in design and pedagogy processes, and there are clear common 
elements among quality measures. These are typically comprised of a combination of course navigation 
and information, course design, accessibility, technology/tools, content, activities, and assessments, 
interaction, and feedback (McGahan, Jackson, & Premer, 2015; OLC, n.d.). 


When designing an online course, these elements may be planned for in advance, and quality 
evaluation and assurance provided. It should be noted that even courses designed for online learning 
should be assessed regularly and continually improved to ensure best practices for quality. In an ERT 
situation, it is helpful for teachers to be familiar with quality standards for online learning, but the focus 
during these events is often just to move forward. Following the initial point of crisis, it may be possible 
but is not always the best solution to convert an ERT course into an online course, and it may be more 
efficient and effective to redesign the course with online quality standards in mind (O’Keefe, Rafferty, 
Gunder, & Vignare, 2020). 


Communication 


Communication, and planning for communication, is important in any course, and is critical in online 
courses, where students and instructors do not necessarily interact synchronously on a regular basis. 
Research has clearly linked high-quality communication between both students and instructors as well 
as students and their peers to higher levels of course engagement, positive outcomes, and student 
satisfaction (Jaggars & Xu, 2016; Leh & Burk, 2016). 


Effective communication in online courses is marked by opportunities for students to connect both 
individually and in groups, and with each other and their instructor(s) in an open, inclusive, and 
equitable environment. In online course design, these opportunities may range from welcome 
messages and icebreaker activities to synchronous and asynchronous communication through various 
media (i.e., video, discussion boards, etc.), and include a variety of instructional activities such as 
discussion, peer critique/review, group projects, etc. (Leh & Burk, 2016; O’Keefe, Rafferty, Gunder, & 
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Vignare, 2020; Roddy, et. al., 2017). In an ERT situation, particular care should be taken to ensure that 
course changes and expectations are clear, that students are able to contact the instructor, and that the 
change of format from either on-campus or blended to fully remote includes communication 
opportunities that are accessible to all students. In both planned online learning experiences and ERT 
environments, the communication focus should remain on creating an open and equitable community, 
and in translating the human elements of communication in teaching and learning to the online 
environment. 


Technology 


The rapid advancement of educational technology over the past two decades has made it possible to 
provide highly engaging, accessible, and effective online learning experiences to a wide audience of 
students. However, it has also brought an expansive catalog of possibilities that can be challenging to 
navigate when designing an online course, and even more so when shifting to ERT. In either situation, it 
is key to bear in mind that technology should be selected to fit course needs and learning outcomes, 
and the success of technological tools will depend upon their implementation (Hodges, Moore, Lockee, 
Trust, & Bond, 2020; O’Keefe, Rafferty, Gunder, & Vignare, 2020). 


Typically, the learning management system (LMS) is the primary technological tool for delivering online 
or remote instruction. In addition to the LMS, there are many options, both technology and digital 
courseware, that provide additional choices for content delivery, engagement, and assessment. 
Primary considerations for selecting technology tools and digital courseware in both online learning 
and ERT, beyond ensuring that the tools selected meet the needs of the course, include consideration of 
asynchronous and synchronous capabilities, any associated costs to either the institution or student, 
and how the tool will be integrated into the course (via LMS or independently) and supported (O’Keefe, 
Rafferty, Gunder, & Vignare, 2020; Roddy, et al., 2017). In an emergent situation, it is particularly 
important to consider the flexibility needs of students, as well as the potential costs, including financial, 
time and effort to learn the tool(s), and reliable access to the tool(s) (O’Keefe, Rafferty, Gunder, & 
Vignare, 2020). 


Student Support 


Student support is critical to student success in all learning environments, but requires dedicated 
forethought for online learners, as they may not have access to on-campus resources (Roddy et al., 
2017). As Roddy et al. synthesized, there are four key areas for supporting online students: academic 
support, technology assistance, health and well-being resources, and a sense of belonging and/or 
community. 


Academic support includes the aforementioned student-instructor interaction, orientation services, 
appropriate scaffolding for effective online learning, and access to resources such as tutoring and 
library services (O’Keefe, Rafferty, Gunder, & Vignare, 2020; Roddy et al., 2017). Additionally, any 
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resources specific to the course should be provided in online-accessible options. Technology assistance 
includes proactive support in selecting user-friendly and flexible options and providing training 
opportunities up front as well as access to ongoing technical support, as needed. Health and well-being 
services for online students can be challenging; there are some barriers with regard to geographical 
location and even funding models for student fees that potentially limit access to such services. 
However, there are options for providing care and referral services, and technology has enhanced the 
availability of such services - through both institutions and third-party providers. It is important for 
teachers to be open, available, and observant of potential needs in order to optimally support student 
needs for health and well-being services. Finally, establishing belonging and community is considered 
an essential support for online students. Communication among students and peers as well as 
instructors is critical, and it is most effective to both ensure that these interactions are high-quality and 
infused with human connection. 


In an ERT situation, it is not always possible to quickly provide all of these areas of student support. In 
such environments, reviewing critical needs for course success (such as library services/access, for 
example) can guide prioritization, and teachers communication and awareness of indicators of student 
needs are the most important considerations (O’ Keefe, Rafferty, Gunder, & Vignare, 2020). 


Teacher Support 


In addition to student support, teacher support is essential for successful experiences in both online 
teaching and learning as well as ERT (Serianni, 2020). The transition to successful online teaching and 
learning includes the acquisition of both pedagogical and technical skills suited to the online 
environment along with an understanding of the needs of online learners (Roddy et al., 2017; Serianni, 
2020). Though many skills do translate from other modalities to online teaching, some specific 
competencies identified in the literature for online environments include effective communication 
skills, adequate technological/technical skills, and proficiency with monitoring student progress, 
providing student feedback, and appropriate follow-up are essential to reduce student disengagement. 
In addition to these individual needs, it is important for teachers to have access to instructional design 
support and technology support and/or training (O’Keefe, Rafferty, Gunder, & Vignare, 2020). 


In ERT situations, not only do teachers not have the time to engage in support services prior to teaching 
in the online environment, they may have limited or no access to services that are normally provided 
(O’Keefe, Rafferty, Gunder, & Vignare, 2020). Many organizations and institutions may gather self-serve 
resources in these situations, and in addition to these, teachers should attend to recommendations 
noted previously regarding course design, content, assessment, technology, and support. 
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Appendix VIII: Survey Protocols 
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K-12 Survey 


Introduction 

Thank you for your participation in this study. The purpose is to conduct a developmental study to 
understand the national position to provide quality online learning across institutions during the 
COVID-19 pandemic in the Kingdom of Saudi Arabia (KSA) and identify areas for future investment 
and improvement. The resulting recommendations will provide information to support capacity and 
quality in online education throughout the kingdom to enhance the potential for Vision 2030 
excellence in a diversified and knowledge-based economy. 


* What best describes your role? 


©™~ Administrator 
«Staff 
Teacher 


* Parent 


©™* Student 
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Directions 

In this section, questions are designed to understand the perceptions and experiences of 
administrators, staff, and teachers with regard to online education. Please respond to these items as 
they relate to your role, and select 'N/A’ for any items that do not apply to your role or experience. The 
first block of questions is for prior to COVID-19 and the second block is for after COVID-19 influenced 
the state of learning. 
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Leadership: Governance - Admin/Staff/Teachers 
The leadership is accountable to a governance body and is responsible for setting and meeting the 
operational and strategic goals in support of the online education mission and vision statements. 


Please rate your agreement with each of the following items prior to COVID-19. 


Strongly 
Disagree Disagree Neutral Agree Strongly Agree NA 


The governance 

structure enables clear, 

effective, and rN om ion eS co a 
comprehensive decision 1S ‘ S \ \ \ 
making related to online 

education. 


The governance 

structure incorporates a 

diverse body of 

decision-makers, wo oo co CO oO @ 
including teachers and ‘ . 
staff, and those who are 
knowledgeable about K- 
12 online learning. 


The governance 

structure requires and 

enables systematic and 7 é Co = co € 
continuous improvement > 
telated to online 
education. 
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Please rate your agreement with each of the following items after COVID-19 influenced the state of 
learning. 


Strongly 
Disagree Disagree Neutral Agree Strongly Agree NIA 


The governance 

structure enables clear, 

effective, and [o— a [= c) Fam * 
comprehensive decision : . 
making related to online 
education. 


The governance 

structure incorporates a 

diverse body of 

decision-makers, cn oO cy a oO oe 
including teachers and x ; 
staff, and those who are 
knowledgeable about K- 

12 online learning. 


The governance 

structure requires and 

enables systematic and ~ eo de a ~ r 
continuous improvement Bs 
related to online 
education. 
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Leadership: Strategies - Admin/Staff/Teachers 
The leadership is accountable to a governance body and is responsible for setting and meeting the 
operational and strategic goals in support of the online education mission and vision statements. 
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Please rate your agreement with each of the following items prior to COVID-19. 


Amission, vision, and 
strategic plan is 
developed addressing 
the importance of quality 
in online learning. 


Astrategy for online 
learning is articulated 
and updated regularly 
incorporating emerging 
educational and 
business trends and 
requirements. 


The strategic plan tor 
online learning aligns 
with other strategic plans 
(e.g., academic, 
technology) within the 
school or district. 


Leadership has 
developed goals that 
align with the strategy 
for online learning. 


Leadership identifies 
measures of goals and 
collects timely data to 
ensure quality and 
identify areas of 
improvement. 


Strongly 
Disagree 


Disagree 


¢ 


C) 


Neutral 


Agree 


C) 


a 


Strongly Agree 


cy 


O 


NA 


Please rate your agreement with each of the following items after COVID-19 influenced the state of 


learning. 


Strongly 
Disagree 


A mission, vision, and 

strategic plan is 

developed addressing H 
the importance of quality 

in online learning. 


A strategy for online 

learning is articulated 

and updated regularly 
incorporating emerging \ 
educational and 

business trends and 
requirements. 


The strategic plan for 
online learning aligns 


with other strategic plans cr 


(e.g., academic, 
technology) within the 
school or district. 


Leadership has 

developed goals that rs 
align with the strategy 

for online learning. 


Leadership identifies 
measures of goals and 


collects timely data to ran 


ensure quality and 
identify areas of 
improvement. 
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Leadership: Policies - Admin/Staff/Teachers 


The leadership is accountable to a governance body and is responsible for setting and meeting the 
operational and strategic goals in support of the online education mission and vision statements. 


Agree 
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Strongly Agree 


N/A 
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Please rate your agreement with each of the following items prior to COVID-19. 


Strongly 

Disagree Disagree Neutral Agree Strongly Agree NA 
Policies and procedures 
for online learning are vom Fon oN \ 
reviewed and updated \ C S ® S C 
regularly. 
Policies encourage the 
development of online 
classes and programs ~ my + ry ~ ~ 

— ( ( C ( 4 ( 


(e.g., a policy for 
intellectual property of 
class materials). 


Policies address 

academic integrity of 

online class cr’ & & @ £ ‘oe } 
assignments and 

assessments, 


Promotion policies of 

teachers and staff 

include criteria co oO o e [ @ 
supporting online . os . 
teaching, learning, and 
innovation. 


Please rate your agreement with each of the following items after COVID-19 influenced the state of 
learning. 


Strongly 
Disagree Disagree Neutral Agree Strongly Agree N/A 


Policies and procedures 

for online learning are rc re CO rX ' 
reviewed and updated © XV S 2 
regularly. 


Policies encourage the 

development of online 

classes and programs oO a lam Cc oO a 
(e.g., a policy for ; . . 
intellectual property of 
class materials). 


ya 
va 


Policies address 

academic integrity of Fa Cc) ry e rn ‘eo 
class assignments and ‘ \ 
assessments. 


Promotion policies of 

teachers and statft 

include criteria Pao om 
supporting online 
teaching, learning, and 
innovation. 


8, 
O 
; 


'e 


= 


200 _—_—eéO——E,EE—_—_—_—_—_—_——LLOL— 


K-12 Survey 


Leadership: Process - Admin/Staff/Teachers 


The leadership is accountable to a governance body and is responsible for setting and meeting the 
operational and strategic goals in support of the online education mission and vision statements. 


Please rate your agreement with each of the following items prior to COVID-19. 


Processes exist to 
provide professional 
development, training, 
support, and evaluation 
of online learning. 


Evaluations are 
conducted periodically 
by qualified and 
objective external parties 
who have a 
demonstrated ability to 
deliver a comprehensive 
review of internal 
processes. 


A process for planning 
and allocating resources 
for online learning, 
including financial 
resources, in 
accordance with the 
strategic plan is 
established. 


Strongly 
Disagree 
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Disagree 


Neutral 
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Agree 


Strongly Agree 


N/A 
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Please rate your agreement with each of the following items after COVID-19 influenced the state of 
learning. 


Strongly 
Disagree Disagree Neutral Agree Strongly Agree NIA 


Processes exist to 

provide professional 

development, training, ‘ e C 1 
support, and evaluation 

of online learning. 


Evaluations are 
conducted periodically 
by qualified and 
objective external parties 
who have a 
demonstrated ability to 
deliver a comprehensive 
review of internal 
processes. 


A process for planning 

and allocating resources 

for online learning, 

including financial cr o~ a rc rN ry 
resources, in ty : ‘ 
accordance with the 

strategic plan is 

established. 
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Leadership: Resource Allocation - Admin/Staff/Teachers 
The leadership is accountable to a governance body and is responsible for setting and meeting the 
operational and strategic goals in support of the online education mission and vision statements. 
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Please rate your agreement with each of the following items prior to COVID-19. 


Strongly 
Disagree Disagree Neutral Agree Strongly Agree NIA 


Sufficient resource 

allocation, including 

financial resources, to 

effectively support the C) () * C) @ () 
mission of online ; ; ; 
education is 

demonstrated. 


Incentives and/or 

financial compensation 

are provided to teachers 

and staff for professional €) Cj ‘ea oO QO @ 
development and 

instructional design 

activities. 


Resources are 

adequately allocated 

and projected to sustain C) O C CO e C) 
and scale online : : ; 
learning. 


An investment is made 

in instructional personnel 

and staff time to support Pot ‘ie an cm 
online learning and - “- we 
instructional 

technologies. 
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Please rate your agreement with each of the following items after COVID-19 influenced the state of 


learning. 
Strongly 
Disagree Disagree Neutral Agree Strongly Agree NIA 
Sufficient resource 
allocation, including 
financial resources, to 
effectively support the ¢ ( * () eo ( \ 
mission of online 
education is 
demonstrated. 


Incentives and/or 

financial compensation 

are provided to teachers 

and staff for professional C> (* Cy C ES ce) 
development and 

instructional design 

activities. 


Resources are 

adequately allocated 

and projected to sustain £ (~ r~ ea = @ 
and scale online 

learning. 


An investment is made 

in instructional personnel 

and staff time to support Cc co o an co cf 
online learning and ‘ee = - \ - 
instructional 

technologies. 
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Leadership: Staffing/Innovation - Admin/Staff/Teachers 
The leadership is accountable to a governance body and is responsible for setting and meeting the 
operational and strategic goals in support of the online education mission and vision statements. 
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Please rate your agreement with each of the following items prior to COVID-19. 


Strongly 


Disagree Disagree 


Asenior level academic 

as, " oe ~ 
position leads online 4 C ; 
learning. 


Sufficient qualified 

professional, 

administrative, and ry O 
support staff are ? 

provided to meet online 

learning strategic goals. 


Technical statt are 

provided to support the 

learning environment Van e 
and the technology ; 

needs of students and 

teachers. 


Leadership incorporates 
an expectation of 


innovation in teaching fo om 


and leaming through 
staff evaluations and 
promotions. 


Neutral 


co 


Agree 


@ 


Strongly Agree 


va 


N/A 


Please rate your agreement with each of the following items after COVID-19 influenced the state of 


learning. 


Strongly 


Disagree Disagree 


Asenior level academic 


position leads online (3 C) 


learning. 


Sufficient qualified 

protessional, 

administrative, and Co om 
support staff are 
provided to meet online 
learning strategic goals. 


Technical staff are 
provided to support the 


learning environment cr C + 


and the technology * 
needs of students and 
teachers. 


Leadership incorporates 
an expectation of 


innovation in teaching rc r> 


and leaming through 
staff evaluations and 
promotions, 


Agree 


® 


Strongly Agree 


, 


N/A 


205 


206 


ONLINE LEARNING 
CONSORTIUM 


yiball Ajo 1 ONLINE LEARNING 
Hau a_i (Q} Cleaweca niu 
National eLeaming Cente 


K-12 Survey 


Curriculum Design & Planning: Instructional Design Methods - Admin/Staff/Teachers 

A quality online course adopts and implements instructional design methods that enable effective 
online instruction for both institutionally developed courses as well as licensed content from other 
sources. 


Please rate your agreement with each of the following items prior to COVID-19. 


Strongly 

Disagree Disagree Neutral Agree Strongly Agree NIA 
Implementation of 
curriculum developed by 
the Ministry of Education & @ * C) ( ( 
is a core responsibility : : i - : 
for teachers. 
The instructional design 
of online classes is 
supported by teachers, co eo oo a a eS 


instructional designers, ; . 
technical experts, and 
evaluators. 


Please rate your agreement with each of the following items after COVID-19 influenced the state of 
learning. 


Strongly 

Disagree Disagree Neutral Agree Strongly Agree NIA 
Implementation of 
curriculum developed by 7 
the Ministry of Education £ g cr ‘ei c ry 


is a core responsibility 
for teachers. 


The instructional design 

of classes is supported 

by teachers, instructional & a ra C) C* C 
designers, technical 

experts, and evaluators. 
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Curriculum Design & Planning: Instructional Design Methods - Teachers 

A quality online course adopts and implements instructional design methods that enable effective 
online instruction for both institutionally developed courses as well as licensed content from other 
sources. 


Please rate your agreement with each of the following items prior to COVID-19. 


Strongly 
Disagree Disagree Neutral Agree Strongly Agree NIA 


The instructional design 

focuses on authentic and 

active learning that is 

student-centered fo a r~ a co lan 
creating opportunities for i \- . x 

students to interact with 

each other and the 

teacher. 


The instructional design 

includes a consistent 

class structure, 

manageable units, and Co co Cc 

similar organization © \ CO \ © 
techniques across 

classes for coherency of 

student experience. 


Please rate your agreement with each of the following items after COVID-19 influenced the state of 
learning. 


Strongly 
Disagree Disagree Neutral Agree Strongly Agree NIA 


The instructional design 

focuses on authentic and 

active learning that is 

student-centered ry *@ ry * va Cc) 
creating opportunities for 
students to interact with 
each other and the 
teacher. 


The instructional design 

includes a consistent 

class structure, 

manageable units, and Fan "a rn oO Fa oO 
similar organization s - \ ‘ ; 
techniques across 

classes for coherency of 

student experience. 
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Curriculum Design & Planning: Universal Design for Learning - Admin/Staff/Teachers 

A quality online course adopts and implements instructional design methods that enable effective 
online instruction for both institutionally developed courses as well as licensed content from other 
sources. 


Please rate your agreement with each of the following items prior to COVID-19. 


Strongly 
Disagree Disagree Neutral Agree Strongly Agree NIA 


Instructional materials 

are quality materials and 

accessible to all rc ro rc — ot e 
students, easy to use, ‘ 
and accessible on any 
Internet-ready device. 


Online classes are 

designed based on 

research-design 

principles and rc? co ry Po ry a 
accessibility guidelines ‘ : , 
with consideration for 

usability, including ease 

of navigation. 


Please rate your agreement with each of the following items after COVID-19 influenced the state of 
learning. 


Strongly 
Disagree Disagree Neutral Agree Strongly Agree N/A 


Instructional materials 

are quality materials and 

accessible to all co - rf Po Co e 
students, easy to use, te : : 
and accessible on any 
Internet-ready device. 


Online classes are 

designed based on 

research-design 

principles and Cc Cc Co a co a 
accessibility guidelines * ‘ % : 
with consideration for 
usability, including ease 
of navigation. 
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Curriculum Design & Planning: Learning Objectives - Admin/Staff/Teachers 

A quality online course adopts and implements instructional design methods that enable effective 
online instruction for both institutionally developed courses as well as licensed content from other 
sources. 


Please rate your agreement with each of the following items prior to COVID-19. 


Strongly 
Disagree Disagree Neutral Agree Strongly Agree NiA 


Measurable learning 
objectives are provided 
for each online class and 
clearly state what the co « e a c~ a 
learner will be able to > ss : . 
demonstrate as a result 

of successfully 

completing the class. 


A development process 

is followed that ensures 

online classes are 

designed so that 

students develop the CS (> . C) ‘* C) 
necessary knowledge 

and skills to meet 

measurable learning 

objectives. 
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Please rate your agreement with each of the following items after COVID-19 influenced the state of 
learning. 


Strongly 
Disagree Disagree Neutral Agree Strongly Agree NIA 


Measurable learning 

objectives are provided 

for each class and 

clearly state what the - * co i rc — 
learner will be able to : : ‘ \ 

demonstrate as a result 
of successfully 
completing the class. 


A development process 

is followed that ensures 

classes are designed so 

that students develop fo ~ : on 

the necessary * © ‘ + 
knowledge and skills to 
meet measurable 
learning objectives. 


arcane © _ [el peace 
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Curriculum Design & Planning: Alignment with Standards - Admin/Staff/Teachers 

A quality online course adopts and implements instructional design methods that enable effective 
online instruction for both institutionally developed courses as well as licensed content from other 
sources. 


Please rate your agreement with each of the following items prior to COVID-19. 


Strongly 
Disagree Disagree Neutral Agree Strongly Agree NIA 


Guidelines regarding 

standards for class 

development, and 

instructional design and cf ~~ co 7 ~~ 7 
delivery of online ; 
classes (e.g., learning 
materials, assessment 
strategies) are used. 
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Please rate your agreement with each of the following items after COVID-19 influenced the state of 


learning. 


Strongly 
Disagree 


Guidelines regarding 

standards for class 

development, and 

instructional design and fo" 
delivery of online ; 
classes (e.g., learning 

materials, assessment 

strategies) are used. 


Curriculum Design & Planning: Alignment with Standards - Teachers 
A quality online course adopts and implements instructional design methods that enable effective 
online instruction for both institutionally developed courses as well as licensed content from other 


sources. 


Please rate your agreement with each of the following items prior to COVID-19. 


Strongly 
Disagree 


Content and learning 

activities align with the a 
assessments and 
learning objectives. 


Disagree 


Disagree 


Neutral 
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Neutral 


Agree 
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Agree 


c 


Strongly Agree 


Strongly Agree 


N/A 


NIA 


Please rate your agreement with each of the following items after COVID-19 influenced the state of 


learning. 


Strongly 
Disagree 


Content and learning 

activities align with the [- 
assessments and . 
learning objectives. 


Disagree 


e 


Neutral 


Agree 


CO 
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Strongly Agree 


NIA 
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Curriculum Design & Planning: Class Materials & Content - Teachers 

A quality online course adopts and implements instructional design methods that enable effective 
online instruction for both institutionally developed courses as well as licensed content from other 
sources. 


Please rate your agreement with each of the following items prior to COVID-19. 


Strongly 
Disagree Disagree Neutral Agree Strongly Agree N/A 


Online classes include a 

syllabus or a document 

on the class site 

outlining learning 

objectives, assessment & () & S = Cy 
methods, and 

descriptions of learning 

activities, including 

content. 


Content, including 

reading assignments 

and lectures, aligns with ) C) e C C2) a 
the assessments and ; ; 

learning objectives. 


Content used includes 

materials that are 

accessible and lan ~ r . r + 
interactive, such as > C C C 
Open Education 

Resource (OER). 


Online classes include 

content that is current, o is co O cC a 
tich, and sufficient in ‘ ‘ - \ 
breadth and depth. 
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Please rate your agreement with each of the following items after COVID-19 influenced the state of 
learning. 


Strongly 
Disagree Disagree Neutral Agree Strongly Agree NIA 


Online classes include a 

syllabus or a document 

on the class site 

outlining learning 

objectives, assessment & C & ‘é & C 
methods, and 

descriptions of learning 

activities, including 

content. 


Content, including 

reading assignments 

and lectures, aligns with / ( a ( C3 ‘e 
the assessments and 

learning objectives. 


Content used includes 

materials that are 

accessible and oa e > _ ra eo 
interactive, such as > : 
Open Education 

Resource (OER). 


Online classes include 


content that is current, cr ( \ oo o on oO 
rich, and sufficient in : : hs \ = NL 
breadth and depth. 
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Curriculum Design & Planning: Innovation - Admin/Staff/Teachers 

A quality online course adopts and implements instructional design methods that enable effective 
online instruction for both institutionally developed courses as well as licensed content from other 
sources. 
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Please rate your agreement with each of the following items prior to COVID-19. 


Strongly 
Disagree Disagree Neutral Agree Strongly Agree N/A 


Teachers and staff are 
supported and 
encouraged to 
incorporate pedagogical 
and technological 
methods to improve 
student outcomes 
through curricular 
redesign. 


ae 
a" 
) 
. 
~ 
ao 


Please rate your agreement with each of the following items after COVID-19 influenced the state of 
learning. 


Strongly 
Disagree Disagree Neutral Agree Strongly Agree NIA 


Teachers and staff are 

supported and 

encouraged to 

incorporate pedagogical 

and technological @ ( 2 e f~ (\ 
methods to improve : Z 7 y 
student outcomes 

through curricular 

redesign. 
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Online Teaching & Learning: Communication - Admin/Staff/Teachers 


Online teachers support learning and facilitates presence (teacher, social, and learner) with digital 
pedagogy, communication, and engagement. 


Please rate your agreement with each of the following items prior to COVID-19. 


Strongly 
Disagree Disagree Neutral Agree Strongly Agree NIA 


Teachers are expected 

to hold virtual 

synchronous check-ins 

with students and 

frequently communicate o Cc fo e e ry 
with their students via . . 
email, class news and 
announcements, online 
discussions, and 
assignment feedback. 


Clear standards are 

established for teacher 
communications 

concerning online { 
teaching (e.g. response 

time, contact 

information, etc.). 


Please rate your agreement with each of the following items after COVID-19 influenced the state of 
learning. 


Strongly 
Disagree Disagree Neutral Agree Strongly Agree NA 


Teachers are expected 
to hold virtual 
synchronous check-ins 
with students and 
frequently communicate 
with their students via 
email, class news and 
announcements, online 
discussions, and 
assignment feedback. 


= 
e 
: 
’' 
z 


Clear standards are 

established for teacher 

communications 

concerning online C CO ‘a ‘oe rc 7 
teaching (e.g. response 

time, contact 

information, etc.). 
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Online Teaching & Learning: Engagement/Expectation Setting - Admin/Staff/Teachers 
Online teachers support learning and facilitates presence (teacher, social, and learner) with digital 
pedagogy, communication, and engagement. 


Please rate your agreement with each of the following items prior to COVID-19. 


Strongly 
Disagree Disagree Neutral Agree Strongly Agree NIA 


Online class design 

promotes student 

engagement, including 
academic challenge and { 
social involvement with 

each other and the 

instructor. 


Online classes contain 

clear, concise, and Co - Co oO nes van 
useful assignment ‘ \ 
instructions. 


Please rate your agreement with each of the following items after COVID-19 influenced the state of 
learning. 


Strongly 
Disagree Disagree Neutral Agree Strongly Agree NIA 


Online class design 

promotes student 

engagement, including 

academic challenge and { & : ‘@ { ( 
social involvement with 

each other and the 

instructor. 


Online classes contain 


clear, concise, and o Cc o CO os Cc 
useful assignment x \ RK 
instructions. 
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Online Teaching & Learning: Expectation Setting - Teachers 
Online teachers support learning and facilitates presence (teacher, social, and learner) with digital 
pedagogy, communication, and engagement. 
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Please rate your agreement with each of the following items prior to COVID-19. 


Strongly 


Disagree Disagree Neutral 


Expectations for student 
assignment completion, 
including class schedule 


with deadlines, grading Co 7 mo 


scheme and associated 
policies, and teachers' 
feedback are provided in 
the class syllabus. 


Barriers to learning are 
reduced in the online 
classes by including 
clear expectations of 
student activity 
(participation and 
performance). 


Agree Strongly Agree NIA 
C 2 & 
C oe CC 


Please rate your agreement with each of the following items after COVID-19 influenced the state of 


learning. 


Strongly 


Disagree Disagree Neutral 


Expectations for student 
assignment completion, 

including class schedule 

with deadlines, grading fo ea fo 
scheme and associated \- > 
policies, and teachers’ 

feedback are provided in 

the class syllabus. 


Barriers to learning are 

reduced in the online 

classes by including 

clear expectations of ff : 4 
student activity 

(participation and 

performance). 
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rs a 
( © ¢ 


ONLINE LEARNING 


ONSORTIUM 


Online Teaching & Learning: Outcomes - Admin/Staff/Teachers 


Online teachers support learning and facilitates presence (teacher, social, and learner) with digital 


pedagogy, communication, and engagement. 


217 


\ ONLINE LEARNING 


Please rate your agreement with each of the following items prior to COVID-19. 


Strongly 

Disagree Disagree Neutral Agree Strongly Agree NIA 
Online classes have _ 
comparable outcomes to 4 C 4 : ( | s : oo 
onsite classes. 


Please rate your agreement with each of the following items after COVID-19 influenced the state of 
learning. 


Strongly 
Disagree Disagree Neutral Agree Strongly Agree NIA 


Online classes have 

~ = . ~ 
comparable outcomes to { C ~ { | ‘a ‘a \ 
onsite classes. 


Online Teaching & Learning: Outcomes - Teachers 


Online teachers support learning and facilitates presence (teacher, social, and learner) with digital 
pedagogy, communication, and engagement. 


Please rate your agreement with each of the following items prior to COVID-19. 


Strongly 

Disagree Disagree Neutral Agree Strongly Agree N/A 
Online classes are 
designed with specific 
measurable learning r ( cr va rc r 


objectives and aligned 
with assessments and 
learning activities. 


Please rate your agreement with each of the following items after COVID-19 influenced the state of 


learning. 
Strongly 
Disagree Disagree Neutral Agree Strongly Agree NIA 
Online classes are 
designed with specific 
measurable learning va Cc ro ae pn co 
objectives and aligned ‘ . ; 


with assessments and 
learning activities. 
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Online Teaching & Learning: Class Interaction - Admin/Staff/Teachers 


Online teachers support learning and facilitates presence (teacher, social, and learner) with digital 
pedagogy, communication, and engagement. 


Please rate your agreement with each of the following items prior to COVID-19. 


Strongly 

Disagree Disagree Neutral Agree Strongly Agree NA 
Student-to-student and 
teacher-to-student 
interactions are a ¢ ro ¢ cr ¢ 
essential characteristics 


that are encouraged and 
facilitated. 


Please rate your agreement with each of the following items after COVID-19 influenced the state of 
learning. 


Strongly 

Disagree Disagree Neutral Agree Strongly Agree NIA 
Student-to-student and 
teacher-to-student 
interactions are c~ . eo a co e 
essential characteristics > - - : s . 


that are encouraged and 
facilitated. 
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Online Teaching & Learning: Class Interaction - Teachers 


Online teachers support learning and facilitates presence (teacher, social, and learner) with digital 
pedagogy, communication, and engagement. 
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Please rate your agreement with each of the following items prior to COVID-19. 


Strongly 

Disagree Disagree Neutral Agree Strongly Agree N/A 
Student interactions with 
other students and with 
the instructor are & { ' # C) = C% 
facilitated in a variety of 
ways. 


Students participate in 7 7 7 
meaningful work groups e (> : C) C C* 
promoting collaboration. 


Peer interaction 

facilitates active learning 

through frequent and EF ( 1 ( )\ = (% 
ongoing peer- 

involvement. 


Please rate your agreement with each of the following items after COVID-19 influenced the state of 


learning. 
Strongly 
Disagree Disagree Neutral Agree Strongly Agree NIA 
Student interactions with 
other students and with 7 7 
the instructor are ¢ ( & C1 « ( % 


facilitated in a variety of 
ways. 


Students participate in 
meaningful work groups ( C 
promoting collaboration. 


Peer interaction 

facilitates active learning 

through frequent and # s & CO) = (\ 
ongoing peer- 

involvement. 
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Online Teaching & Learning: Feedback - Admin/Staff/Teachers 
Online teachers support learning and facilitates presence (teacher, social, and learner) with digital 
pedagogy, communication, and engagement. 
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Please rate your agreement with each of the following items prior to COVID-19. 


Strongly 

Disagree Disagree Neutral Agree Strongly Agree NIA 
Teachers provide timely 
and detailed feedback r~ a 


on assessments and 
student inquiries. 


Please rate your agreement with each of the following items after COVID-19 influenced the state of 
learning. 


Strongly 

Disagree Disagree Neutral Agree Strongly Agree NIA 
Teachers provide timely 
and detailed feedback a: a co S cm a 
on assessments and = : : : 


student inquiries. 


HE ONLINE LEARNING 
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Online Teaching & Learning: Feedback - Teachers 


Online teachers support learning and facilitates presence (teacher, social, and learner) with digital 


pedagogy, communication, and engagement. 


Please rate your agreement with each of the following items prior to COVID-19. 


Strongly 

Disagree Disagree Neutral Agree Strongly Agree NIA 
Feedback on student 
assignments and 
questions is constructive & & c C -_ (~ 
and provided in a timely 
manner. 


Please rate your agreement with each of the following items after COVID-19 influenced the state of 
learning. 


Strongly 

Disagree Disagree Neutral Agree Strongly Agree NA 
Feedback on student 
assignments and 
questions is constructive Es ( = ( ¢ € 
and provided in a timely 
manner. 
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Online Teaching & Learning: Innovation - Admin/Staff/Teachers 


Online teachers support learning and facilitates presence (teacher, social, and learner) with digital 
pedagogy, communication, and engagement. 


Please rate your agreement with each of the following items prior to COVID-19. 


Strongly 

Disagree Disagree Neutral Agree Strongly Agree NIA 
Teachers and staff are 
encouraged to innovate - 
in online teaching and in 
their support of students. 


Please rate your agreement with each of the following items after COVID-19 influenced the state of 


learning. 
Strongly 
Disagree Disagree Neutral Agree Strongly Agree NIA 
Teachers and staff are 
encouraged to innovate cr ¢ ro a rr? a 


in online teaching and in 
their support of students. 
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Assessment: Strategies - Admin/Staff/Teachers 

Online teachers create and/or implement assessments in online learning environments in ways that 
ensure the validity and reliability of the instruments and procedures. Teachers measure learner 
progress through assessments, projects, and assignments that meet standards-based learning goals, 


and evaluate learner understanding of how these assessments measure achievement of the learning 
objectives. 
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Please rate your agreement with each of the following items prior to COVID-19. 


Strongly 
Disagree Disagree Neutral Agree Strongly Agree N/A 


Online classes have 

multiple timely and 

appropriate activities to 

assess student learning & & & C & C } 
of the aligned learning 

objectives throughout 

the class. 


Please rate your agreement with each of the following items after COVID-19 influenced the state of 
learning. 


Strongly 
Disagree Disagree Neutral Agree Strongly Agree NIA 


Online classes have 

multiple timely and 

appropriate activities to 

assess student learning ¢ ( . C- @ C 
of the aligned learning 

objectives throughout 

the class. 
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Assessment: Strategies - Teachers 


Online teachers create and/or implement assessments in online learning environments in ways that 


ensure the validity and reliability of the instruments and procedures. Teachers measure learner 


progress through assessments, projects, and assignments that meet standards-based learning goals, 
and evaluate learner understanding of how these assessments measure achievement of the learning 


objectives. 
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Please rate your agreement with each of the following items prior to COVID-19. 


Strongly 
Disagree 


Ongoing assessment 
strategies (formative 
assessment) are used to 
measure content 
knowledge, skills, and 
abilities while providing 
students feedback on 
what they have learned 
and on what to improve. 


e 


Students’ self- 

assessments, peer 

review, and feedback H 
opportunities exist 

throughout the course. 


Learning objectives and 
assessment activities 
are closely aligned. 


Disagree 


Neutral 


Agree 


Strongly Agree 


NIA 


Please rate your agreement with each of the following items after COVID-19 influenced the state of 


learning. 


Strongly 
Disagree 


Ongoing assessment 

strategies (formative 
assessment) are used to 
measure content 

knowledge, skills, and 1 
abilities while providing 

students feedback on 

what they have learned 

and on what to improve. 


Students' self- 

assessments, peer 

review, and feedback H 
opportunities exist 

throughout the course. 


Learning objectives and 
assessment activities 4 
are closely aligned. 


Disagree 


Neutral 


Agree 
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N/A 
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Assessment: Processes - Admin/Staff/Teachers 

Online teachers create and/or implement assessments in online learning environments in ways that 
ensure the validity and reliability of the instruments and procedures. Teachers measure learner 
progress through assessments, projects, and assignments that meet standards-based learning goals, 


and evaluate learner understanding of how these assessments measure achievement of the learning 
objectives. 


Please rate your agreement with each of the following items prior to COVID-19. 


Strongly 
Disagree Disagree Neutral Agree Strongly Agree N/A 


Frequent and substantial 

feedback from the 

instructor are provided 

to students through Co — 
assessments in order for 

them to improve their 

learning or target 

remediation. 


Please rate your agreement with each of the following items after COVID-19 influenced the state of 
learning. 


Strongly 
Disagree Disagree Neutral Agree Strongly Agree NA 
Frequent and substantial 
feedback from the 
instructor are provided 
to students through r a r~ r r r 


assessments in order for 
them to improve their 
learning or target 
remediation. 
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Assessment: Processes - Teachers 


Online teachers create and/or implement assessments in online learning environments in ways that 
ensure the validity and reliability of the instruments and procedures. Teachers measure learner 
progress through assessments, projects, and assignments that meet standards-based learning goals, 


and evaluate learner understanding of how these assessments measure achievement of the learning 
objectives. 
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Please rate your agreement with each of the following items prior to COVID-19. 


Strongly 
Disagree Disagree Neutral Agree Strongly Agree N/A 


Regular feedback about 
student performance on 
assessments is provided 
in a timely manner 
throughout the course. 


Assessment criteria are 

clearly described when 

the assignment is issued 

with grading rubrics and cn C lan co la co 
samples of assignments : 
illustrating teacher's 
expectations are 
provided. 


Please rate your agreement with each of the following items after COVID-19 influenced the state of 
learning. 


Strongly 
Disagree Disagree Neutral Agree Strongly Agree NIA 


Regular feedback about 

student performance on 

assessments is provided € ( @ ( y ¢ @ 
in a timely manner 

throughout the course. 


Assessment criteria are 

clearly described when 

the assignment is issued 

with grading rubrics and a - la a - c 
samples of assignments % : : 
illustrating teacher's 
expectations are 
provided. 
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Assessment: Methodology/Innovation - Admin/Staff/Teachers 

Online teachers create and/or implement assessments in online learning environments in ways that 
ensure the validity and reliability of the instruments and procedures. Teachers measure learner 
progress through assessments, projects, and assignments that meet standards-based learning goals, 
and evaluate learner understanding of how these assessments measure achievement of the learning 
objectives. 


Please rate your agreement with each of the following items prior to COVID-19. 


Strongly 
Disagree Disagree Neutral Agree Strongly Agree NIA 


Teachers use an active 

learning assessment 

approach where 

students are not 

assessed solely on o e es a co Cc 
tests/quizzes but are - - ~ a 
provided ample 

opportunity to 

demonstrate proficiency 

in different ways. 


Teachers pilot 

assessment strategies 

that incorporate digital 

literacies and other ‘= C - '@ = oe 
pertinent literacies (e.g., 

blogging, visual 

communication). 


Please rate your agreement with each of the following items after COVID-19 influenced the state of 
learning. 


Strongly 
Disagree Disagree Neutral Agree Strongly Agree NIA 


Teachers use an active 

learning assessment 

approach where 

students are not 

assessed solely on om c Co eo vom 2 
tests/quizzes but are : . 
provided ample 
opportunity to 
demonstrate proficiency 
in different ways. 


Teachers pilot 

assessment strategies 

that incorporate digital 

literacies and other Ce (~ rs C) ‘2 C* 
pertinent literacies (e.g., ; - ; / ; 
blogging, visual 

communication). 
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Technology: Centralized Online Education Infrastructure - Admin 


Educational technology is supported, reliable, accessible, and sufficient to meet the online education 
needs of teachers and students. 


Please rate your agreement with each of the following items prior to COVID-19. 


Strongly 
Disagree Disagree Neutral Agree Strongly Agree NIA 
Adocumented 
technology plan is in 
place to ensure quality 
standards in online 
learning. 


2 ale f * = 


Please rate your agreement with each of the following items after COVID-19 influenced the state of 


learning. 

Strongly 

Disagree Disagree Neutral Agree Strongly Agree NIA 
A documented 
technology plan is in e = = 
place to ensure quality { C- ‘4 C) C (\ 
standards in online 
learning. 
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Technology: Centralized Online Education Infrastructure - Admin/Staff 


Educational technology is supported, reliable, accessible, and sufficient to meet the online education 
needs of teachers and students. 
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Please rate your agreement with each of the following items prior to COVID-19. 


Support for building and 
maintaining the online 
learning technological 
infrastructure is 
addressed by a 
centralized system (e.g., 
Learning Management 
System) with a single 
sign-on (SSO) or single 
identification. 


The Learning 
Management System 
has an interoperability 
protocol established for 
applications to be 
embedded within the 
system for a seamless 
experience. 


Strongly 

Disagree Disagree Neutral Agree Strongly Agree NIA 
C C e * @ C 
4 * C C C 4 


Please rate your agreement with each of the following items after COVID-19 influenced the state of 


learning. 


Support for building and 
maintaining the online 
learning technological 
infrastructure is 
addressed by a 
centralized system (e.g., 
Learning Management 
System) with a single 
sign-on (SSO) or single 
identification. 


The Learning 
Management System 
has an interoperability 
protocol established for 
applications to be 
embedded within the 
system for a seamless 
experience. 


Strongly 

Disagree Disagree Neutral Agree Strongly Agree NIA 
q C- & C q sg 
¢ (~ eS a ¢ oe 
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Technology: Centralized Online Education Infrastructure - Admin/Staff/Teachers 
Educational technology is supported, reliable, accessible, and sufficient to meet the online education 
needs of teachers and students. 


Please rate your agreement with each of the following items prior to COVID-19. 


Strongly 

Disagree Disagree Neutral Agree Strongly Agree N/A 
There is a centralized 
system (e.g., Learning co Cc co lan co ~ 


Management System) 
with a single login. 


Please rate your agreement with each of the following items after COVID-19 influenced the state of 
learning. 


Strongly 

Disagree Disagree Neutral Agree Strongly Agree N/A 
There is a centralized 
system (e.g., Learning 2 e & Cy rt e 


Management System) 
with a single login. 
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Technology: Centralized Online Education Infrastructure - Staff/Teachers 
Educational technology is supported, reliable, accessible, and sufficient to meet the online education 
needs of teachers and students. 


Please rate your agreement with each of the following items prior to COVID-19. 


Strongly 

Disagree Disagree Neutral Agree Strongly Agree NIA 
Online class embedded 
technologies actively 
support the achievement 
of learning outcomes r tc [~ a ¢ Cy 
and delivery of class - An boo ~ A \- 
content, while 


superfluous use of 
technology is minimized. 


Please rate your agreement with each of the following items after COVID-19 influenced the state of 


learning. 


Online class embedded 
technologies actively 
support the achievement 
of learning outcomes 
and delivery of class 
content, while 
superfluous use of 
technology is minimized. 


Strongly 
Disagree 


Disagree 


Neutral 
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Technology: Operability/Modality - Admin/Staff/Teachers 


Educational technology is supported, reliable, accessible, and sufficient to meet the online education 
needs of teachers and students. 


Please rate your agreement with each of the following items prior to COVID-19. 


The Learning 
Management System is 
highly reliable and 
operable. 


Online classes are 
delivered 
asynchronously (over 
time) and synchronously 
(real-time) dependent on 
the learning objectives. 


Blended classes are 
delivered integrating a 
combination of face-to- 
face, online, 
synchronous, and 
asynchronous 
components. 


Strongly 
Disagree 


Disagree 


g 


Neutral 


Agree 


ONLINE LEARNING 
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Agree 


(1 


Strongly Agree 


Strongly Agree 


N/A 


N/A 
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Please rate your agreement with each of the following items after COVID-19 influenced the state of 
learning. 


Strongly 
Disagree Disagree Neutral Agree Strongly Agree NIA 


The Learning 

Management System is cr ‘an re ro a 
highly reliable and ; \ : \ 
operable. 


Online classes are 

delivered 

asynchronously (over o> ‘*. rc 
time) and synchronously 

(real-time) dependent on 

the learning objectives. 


Blended classes are 

delivered integrating a 

combination of face-to- 

face, online, € ® £ CC) & Ci 
synchronous, and 

asynchronous 

components. 
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Technology: Security/ITSM Compliance - Admin 
Educational technology is supported, reliable, accessible, and sufficient to meet the online education 
needs of teachers and students. 


Please rate your agreement with each of the following items prior to COVID-19. 


Strongly 
Disagree Disagree Neutral Agree Strongly Agree N/A 


Digital security 

measures are in place to 

ensure the integrity and { € C4 Lj { } 
validity of information, 

including identity access. 


IT service management 

(ITSM) to design, plan, 

deliver, operate, and 

control information Co eo [o a 7 O 
technology supports the i : 
development and 
delivery of online 
classes and proarams. 
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Please rate your agreement with each of the following items after COVID-19 influenced the state of 
learning. 


Strongly 
Disagree Disagree Neutral Agree Strongly Agree NIA 


Digital security 

measures are in place to 

ensure the integrity and & & # (— & & 
validity of information, : : : : 
including identity access. 


IT service management 

(ITSM) to design, plan, 

deliver, operate, and 

control information \ ee lan cy ~ a 
technology supports the - . 
development and 

delivery of online 

classes and programs. 


vibgll jSpoll 
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Technology: Internet Access/Reliability/Coverage/Innovation - Admin/Staff/Teachers 
Educational technology is supported, reliable, accessible, and sufficient to meet the online education 
needs of teachers and students. 
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Please rate your agreement with each of the following items prior to COVID-19. 


Teachers, staff, and 
students are provided 
with access to the 
internet from any 
location. 


Teachers, staff, and 
students can reliably 
login to core learning 
technology systems trom 
any location. 


Internet access and 
access to core learning 
technologies are 
available from any 
location, 


New and innovative 
technologies are piloted, 
supported, and 
encouraged in online 
classes. 


Strongly 


Disagree 


Disagree 


Neutral 


Fa 


Agree 


Strongly Agree 


N/A 


Please rate your agreement with each of the following items after COVID-19 influenced the state of 


learning. 


Teachers, staff, and 
students are provided 
with access to the 
internet from any 
location. 


Teachers, staff, and 
students can reliably 
login to core learning 
technology systems from 
any location, 


Internet access and 
access to core learning 
technologies are 
available from any 
location. 


New and innovative 
technologies are piloted, 
supported, and 
encouraged in online 
classes. 


Strongly 


Disagree 


Disagree 


e 


Neutral 


Agree 


Strongly Agree 


oN 


NIA 


BOP >iboll Soll 
Sgt gat 
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Student Support: Student Orientation & Support - Admin/Staff/Teachers 
Student support services are available to address the various needs of learners at different levels 
within the organization. The levels of support are appropriate and adequate for learner success. 


Please rate your agreement with each of the following items prior to COVID-19. 


Strongly 
Disagree Disagree Neutral Agree Strongly Agree NIA 


Students are provided 

orientations to their 

online classes, the class -~ c- mo ea = a 
design and activities, : : ss : % 
and the technologies 
used in the class. 


Students are provided 

information on class 

technology and ei (~ ? a iz a 
academic support 

resources. 


Students have access to 

appropriate support 

personnel to address 

technical and academic . & & ® £ s 
questions, problem 

reporting, and 

complaints. 
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Please rate your agreement with each of the following items after COVID-19 influenced the state of 
learning. 


Strongly 
Disagree Disagree Neutral Agree Strongly Agree NIA 


Students are provided 

orientations to their 

online classes, the class co c— vom v- vom a 
design and activities, - : : 
and the technologies 
used in the class. 


Students are provided 

information on class 

technology and ‘& C eS ‘@ > C 
academic support 

resources. 


Students have access to 
appropriate support 

personne! to address 

technical and academic 4 
questions, problem 

reporting, and 

complaints. 


2 pial iSsoll OL falR aaa 
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Student Support: Equity/Accessibility/Compliance Standards - Admin/Staff/Teachers 
Student support services are available to address the various needs of learners at different levels 
within the organization. The levels of support are appropriate and adequate for learner success. 
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Please rate your agreement with each of the following items prior to COVID-19. 


Strongly 
Disagree 


Resources are in place 

to ensure all students, 

including 

underrepresented 

students, have an a 
equitable experience 

(e.g., technology, class 

materials, support help, 

other forms of distance 

education). 


Policy, processes, and 

resources are in place to ~~ 
support students with 

disabilities. 


Online classes 

demonstrate compliance rx 
with accessibility 
standards. 


Disagree Neutral Agree Strongly Agree NIA 
C c * « C 
C ig C * C 
4 ( q g 


Please rate your agreement with each of the following items after COVID-19 influenced the state of 


learning. 


Strongly 
Disagree 


Resources are in place 

to ensure all students, 

including 

underrepresented 

students, have an = 
equitable experience 
(e.g., technology, class 

materials, support help, 

other forms of distance 

education). 


Policy, processes, and 

resources are in place to on 
support students with 
disabilities. 


Online classes 

demonstrate compliance ro 
with accessibility 
standards. 


Disagree Neutral Agree Strongly Agree NIA 
C @ C ¢ 4a 
C m e a Cc 
( c ( * a 


pancreas >. (6) ane 


iif 
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Student Support: Social-Emotional/Innovation - Admin/Staff/Teachers 
Student support services are available to address the various needs of learners at different levels 
within the organization. The levels of support are appropriate and adequate for learner success. 


Please rate your agreement with each of the following items prior to COVID-19. 


Strongly 
Disagree Disagree Neutral Agree Strongly Agree N/A 
Students are provided 
with opportunities for 
social emotional learning 
and support, such as C) & C a & C) 


building relationships, 
culture, and community 
in a remote setting. 


Students are being 

supported in innovative 

ways, such as relevant 

communication modes 

(e.g., text messaging or Fam’ Cc a c mo a 
synchronous chat) or ‘ \ 

through advanced 

communication tools 

(e.g., artificially 

intelligent agents). 


Please rate your agreement with each of the following items after COVID-19 influenced the state of 
learning. 


Strongly 
Disagree Disagree Neutral Agree Strongly Agree NIA 
Students are provided 
with opportunities for 
social emotional learning 
and support, such as - ® & C) £ be 


building relationships, 
culture, and community 
in a remote setting. 


Students are being 

supported in innovative 

ways, such as relevant 

communication modes 

(e.g., text messaging or mo oO Cc oO Fo oO 
synchronous chat) or ‘- ; . b , 
through advanced 

communication tools 

(e.g., artificially 

intelligent agents). 
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Training & Support: Professional Development - Admin/Staff/Teachers 
Training and support is available though mentoring, technical assistance, and timely professional 
development. 


Please rate your agreement with each of the following items prior to COVID-19. 


Strongly 
Disagree Disagree Neutral Agree Strongly Agree NIA 


Teachers receive 

training, assistance, and 

support to prepare for * & ) C\ * & 
classes and teaching 

online. 


Teachers are provided 

ongoing professional 

development related to C (~ ik Cy ‘oe C 
online teaching and 

learning. 


Teachers are provided 

with adequate time to o~ co “ a é co 
engage in professional : * : = Ss 
development. 


Ongoing professional 

development, training, 

and support in pedagogy 

and technology are 

provided for teachers ia C ce Co) tC} C 
and staff to develop the 

knowledge, skills, and 

abilities to design and 

teach online classes. 


Teachers are provided 
orientation/training on 


technology and - =~ ' P 
instructional design used - C - C) = ® 
in the development of 

online classes. 
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Please rate your agreement with each of the following items after COVID-19 influenced the state of 
learning. 


Strongly 
Disagree Disagree Neutral Agree Strongly Agree NIA 


Teachers receive 

training, assistance, and 

support to prepare for & C & ee & 
classes and teaching 

online. 


Teachers are provided 

ongoing professional 

development related to Cc C < C) (~ C ) 
online teaching and 

learning. 


Teachers are provided 

with adequate time to co a co ry co co 
engage in professional : = . ‘ : " 
development. 


Ongoing professional 
development, training, 
and support in pedagogy 
and technology are 
provided for teachers 
and staff to develop the 
knowledge, skills, and 
abilities to design and 
teach online classes. 


Teachers are provided 

orientation/training on 

technology and Co om co rm a -. 
instructional design used 7 > ; > 
in the development of 
online classes. 
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Training & Support: Technical Assistance/Mentoring/Innovation - Admin/Staff/Teachers 
Training and support is available though mentoring, technical assistance, and timely professional 
development. 
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Please rate your agreement with each of the following items prior to COVID-19. 


Strongly 

Disagree Disagree Neutral Agree Strongly Agree N/A 
Technical assistance is 
provided for teachers 
during online class & & & ( 4 q 
development and online 
teaching. 


Peer mentoring 
resources available to ( es f Cc) f C 
online class teachers. 


Teachers are provided 

opportunities to select = ro fo a lo - 
new tools and pilot ; : % 
emerging technologies. 


Please rate your agreement with each of the following items after COVID-19 influenced the state of 
learning. 


Strongly 

Disagree Disagree Neutral Agree Strongly Agree NIA 
Technical assistance is 
provided for teachers 
during online class € C 4 (— # (7 
development and online 
teaching. 


Peer mentoring 
resources available to C C e C) ‘4 C 
online class teachers. 


Teachers are provided 

opportunities to select rr ( r 
new tools and pilot : ; 
emerging technologies. 


K-12 Survey 


Evaluation & Continuous Improvement: Student/Parent Satisfaction - Admin/Teachers 
Evaluation is both internal and external and informs all processes that affect teaching and learning. 
Internal evaluations often are more informal in nature and may provide immediate feedback ona 
targeted area of inquiry. External program evaluations typically look at the entire program from an 
objective perspective that will bring additional credibility to the results. 
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Please rate your agreement with each of the following items prior to COVID-19. 


Strongly 

Disagree Disagree Neutral Agree Strongly Agree N/A 
Aprocess is in place and 
followed for the 
assessment of student 4 ( 4 () 4 ( 


satisfaction with online 
classes and programs. 


A process is in place and 

followed for the 

assessment of parent Cc C ; 4 ic CO C O 
satisfaction with online : : : : : 
classes and programs. 


Please rate your agreement with each of the following items after COVID-19 influenced the state of 


learning. 
Strongly 
Disagree Disagree Neutral Agree Strongly Agree N/A 
A process Is in place and 
followed for the 
assessment of student € C @ C) ¢ (* 


satisfaction with online 
classes and programs. 


A process is in place and 

followed for the 

assessment of parent # 'e ‘- ‘a ‘a C ‘ 
satisfaction with online 

classes and programs. 
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Evaluation & Continuous Improvement: Teacher Satisfaction - Admin/Teachers 

Evaluation is both internal and external and informs all processes that affect teaching and learning. 
Internal evaluations often are more informal in nature and may provide immediate feedback on a 
targeted area of inquiry. External program evaluations typically look at the entire program from an 
objective perspective that will bring additional credibility to the results. 
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Please rate your agreement with each of the following items prior to COVID-19. 


Strongly 

Disagree Disagree Neutral Agree Strongly Agree NIA 
A process is in place and 
followed for the 
assessment of teacher r & & ( q ( 
satisfaction with online 
classes and programs. 


Please rate your agreement with each of the following items after COVID-19 influenced the state of 
learning. 


Strongly 
Disagree Disagree Neutral Agree Strongly Agree NIA 
A process is in place and 
followed for the 
assessment of teacher € & & ¢ 2 
satisfaction with online 
classes and programs. 


i L ONLINE LEARNING 
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Evaluation & Continuous Improvement: Staff Satisfaction - Admin/Staff 

Evaluation is both internal and external and informs all processes that affect teaching and learning. 
Internal evaluations often are more informal in nature and may provide immediate feedback on a 
targeted area of inquiry. External program evaluations typically look at the entire program from an 
objective perspective that will bring additional credibility to the results. 


Please rate your agreement with each of the following items prior to COVID-19. 


Strongly 

Disagree Disagree Neutral Agree Strongly Agree NA 
A process is in place and 
followed for the 
assessment of staff € (~ c (~ r & 
satisfaction with online - : ; : : 
classes and programs. 
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Please rate your agreement with each of the following items after COVID-19 influenced the state of 
learning. 


Strongly 

Disagree Disagree Neutral Agree Strongly Agree NIA 
A process is in place and 
followed for the 
assessment of staff & < - C) é ( \ 


satisfaction with online 
classes and programs. 


Sh, ee! MC}: 


National 
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Evaluation & Continuous Improvement: Class Outcomes & Program Quality - 
Admin/Staff/Teachers 


Evaluation is both internal and external and informs all processes that affect teaching and learning. 
Internal evaluations often are more informal in nature and may provide immediate feedback ona 
targeted area of inquiry. External program evaluations typically look at the entire program from an 
objective perspective that will bring additional credibility to the results. 


Please rate your agreement with each of the following items prior to COVID-19. 


Strongly 
Disagree Disagree Neutral Agree Strongly Agree NIA 


The institution has 

developed a procedure 

and/or policy for 

evaluating and & C € (C Y f— ( \ 
improving online classes ; ; : 7 

and programs ona 

timely basis. 


Intended learning 

outcomes at the class 

and program level are 

reviewed regularly to co 
ensure alignment, 
clarity, utility, 
appropriateness, and 
effectiveness. 


Please rate your agreement with each of the following items after COVID-19 influenced the state of 
learning. 


Strongly 
Disagree Disagree Neutral Agree Strongly Agree NIA 


The institution has 
developed a procedure 
and/or policy for 
evaluating and 
improving online classes 
and programs ona 
timely basis. 


an 
> 
, 
= 
‘ 
= 


Intended learning 

outcomes at the class 

and program level are 

reviewed regularly to o 7 Co _ mo c 
ensure alignment, . : . \ x 
clarity, utility, 

appropriateness, and 

effectiveness. 
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Evaluation & Continuous Improvement: Class Outcomes & Program Quality - Admin 
Evaluation is both internal and external and informs all processes that affect teaching and learning. 
Internal evaluations often are more informal in nature and may provide immediate feedback on a 
targeted area of inquiry. External program evaluations typically look at the entire program from an 
objective perspective that will bring additional credibility to the results. 
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Please rate your agreement with each of the following items prior to COVID-19. 


Online programs are 
assessed through an 
evaluation process that 
applies specific 
established standards. 


A process is in place and 
followed for the 
assessment of student 
academic achievement 
and retention in online 
classes and programs. 


A variety of data 
(academic and 
administrative} are used 
to regularly and 
frequently evaluate 
program effectiveness 
and to guide continuous 
improvement. 


Strongly 
Disagree 


Disagree Neutral Agree Strongly Agree N/A 
( e ( ( ( 
C cs 7 ‘a cC 


— 
rs 
~ 
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Please rate your agreement with each of the following items after COVID-19 influenced the state of 


learning. 


Online programs are 
assessed through an 
evaluation process that 
applies specific 
established standards. 


A process is in place and 
followed for the 
assessment of student 
academic achievement 
and retention in online 
classes and programs. 


A variety of data 
(academic and 
administrative) are used 
to regularly and 
frequently evaluate 
program effectiveness 
and to guide continuous 
improvement. 


Strongly 
Disagree 


Disagree Neutral Agree Strongly Agree NIA 
( f a é 4 
( ‘? ( C C 
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Evaluation & Continuous Improvement: Class Outcomes & Program Quality - Admin/Teachers 
Evaluation is both internal and external and informs all processes that affect teaching and learning. 
Internal evaluations often are more informal in nature and may provide immediate feedback on a 
targeted area of inquiry. External program evaluations typically look at the entire program from an 
objective perspective that will bring additional credibility to the results. 


Please rate your agreement with each of the following items prior to COVID-19. 


Strongly 
Disagree Disagree Neutral Agree Strongly Agree NIA 
A process is in place and 
followed for the 
assessment of teaching & (> m4 ( ) & < 
performance in online ; 
classes. 


Please rate your agreement with each of the following items after COVID-19 influenced the state of 
learning. 


Strongly 

Disagree Disagree Neutral Agree Strongly Agree NA 
Aprocess is in place and 
followed for the 
assessment of teaching = s ey & t C 
performance in online 
classes. 
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Evaluation & Continuous Improvement: Class Outcomes & Program Quality - Admin/Staff 
Evaluation is both internal and external and informs all processes that affect teaching and learning. 
Internal evaluations often are more informal in nature and may provide immediate feedback ona 
targeted area of inquiry. External program evaluations typically look at the entire program from an 
objective perspective that will bring additional credibility to the results. 
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Please rate your agreement with each of the following items prior to COVID-19. 


Strongly 

Disagree Disagree Neutral Agree Strongly Agree N/A 
Aprocess is in place and 
followed for the 
assessment of support 4 ( C () i f 


services for teachers 
and students. 


Please rate your agreement with each of the following items after COVID-19 influenced the state of 


learning. 
Strongly 
Disagree Disagree Neutral Agree Strongly Agree N/A 
A process is in place and 
followed for the 
assessment of support 4 4 = ( a ( } 


services for teachers 
and students. 


DL ONLINE LEARNING 
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Evaluation & Continuous Improvement: Innovation - Admin 

Evaluation is both internal and external and informs all processes that affect teaching and learning. 
Internal evaluations often are more informal in nature and may provide immediate feedback ona 
targeted area of inquiry. External program evaluations typically look at the entire program from an 
objective perspective that will bring additional credibility to the results. 


Please rate your agreement with each of the following items prior to COVID-19. 


Strongly 
Disagree Disagree Neutral Agree Strongly Agree NIA 


Innovative developments 

in analytics guides 

instructional and l- ro co ro mo ~~ 
programmatic decisions ; ; 

about online teaching 

and learning. 
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Please rate your agreement with each of the following items after COVID-19 influenced the state of 
learning. 


Strongly 
Disagree Disagree Neutral Agree Strongly Agree NIA 
Innovative developments 
in analytics guides 
instructional and - a ¢ c 7 4 
programmatic decisions ; 


about online teaching 
and learning. 


CONSORTIUM 
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Directions 


In this section, questions are designed to understand the perceptions and experiences of students 
and parents with regard to online education. Please respond to these items as they relate to your role 
or experience, and select ’N/A’ for any items that do not apply to your role or experience. The first 


block of questions is for prior to COVID-19 and the second block is for after COVID-19 influenced the 
state of learning. 
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Leadership: Governance/Strategies/Policies/Processes/Resource Allocation/Staffing - Parents 
The leadership is accountable to a governance body and is responsible for setting and meeting the 
operational and strategic goals in support of the online education mission and vision statements. 
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Please rate your agreement with each of the following items prior to COVID-19. 


Parents and students 
were asked for their 
opinion about potential 
online learning plans to 
help school leaders 
make decisions. 


School leaders clearly 
communicated their 
goals and plans tor 
online learning to 
parents and students. 


Parents and students 
received school policies 
for online learning (e.g., 
academic integrity, 
online citizenship, and 
appropriate use of the 
internet). 


Class materials 
appeared to comply with 
intellectual property 
guidelines, fair use 
standards, acceptable 
use policies, and 
copyright and licensing 
requirements for third- 
party content. 


Parents and students 
received introductions to 
the online classes and 
technologies to be used. 


Parents and students 
received information on 
where and how they 
could receive support. 


Online learning is 
adequately funded. 


Online learning has 
adequate staft for 
teaching and supporting 
parents and students. 


Strongly 
Disagree 


- 


Disagree 


C) 


C) 


Neutral 


Agree 


Strongly Agree 


cy 


0 


NA 


Please rate your agreement with each of the following items after COVID-19 influenced the state of 


learning. 


Strongly 
Disagree 


Parents and students 

were asked for their 

opinion about potential vom 
online learning plans to 
help school leaders 
make decisions. 


School leaders clearly 
communicated their 

goals and plans for H 
online learning to 

parents and students. 


Parents and students 
received school policies 
for online learning (e.g., 
academic integrity, 
online citizenship, and 
appropriate use of the 
internet). 


Class materials 

appeared to comply with 

intellectual property 

guidelines, fair use 

standards, acceptable ao 
use policies, and 
copyright and licensing 
requirements for third- 
party content. 


Parents and students 

received introductions to 7 
the online classes and ; 
technologies to be used. 


Parents and students 

received information on oo 
where and how they 

could receive support. 


Online learning is cr 
adequately funded. 


Online learning has 

adequate staff for = 
teaching and supporting 

parents and students. 


Disagree 


POP pibaoll jSsoll 


Ga aly oes 


Neutral 


OE C |: 


Agree 


ONLINE LEARNING 
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Strongly Agree 


N/A 
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Curriculum Design & Planning: Instructional Design Methods - Parents 
A quality online course adopts and implements instructional design methods that enable effective 


online instruction for both institutionally developed courses as well as licensed content from other 
sources. 


Please rate your agreement with each of the following items prior to COVID-19. 


Strongly 

Disagree Disagree Neutral Agree Strongly Agree NIA 
The online classes had 
lots of activities where 
students were « & r’ C & ® 
interacting with each 
other and the teacher. 


The online classes were 

organized, and it was 

easy to get around the i C : - C ’) - C ’ 
class sites and find 

things. 


Please rate your agreement with each of the following items after COVID-19 influenced the state of 
learning. 


Strongly 
Disagree Disagree Neutral Agree Strongly Agree NIA 


The online classes had 

lots of activities where 

students were r’ e ® C1 C" ( 
interacting with each 

other and the teacher. 


The online classes were 

organized, and it was 

easy to get around the Lc C ? C ¢ Cr 
class sites and find 

things. 


vial jSsoll 
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Curriculum Design & Planning: Instructional Design Methods - Students 


A quality online course adopts and implements instructional design methods that enable effective 
online instruction for both institutionally developed courses as well as licensed content from other 


252 


sources. 


Please rate your agreement with each of the following items prior to COVID-19. 


Strongly 

Disagree Disagree Neutral Agree Strongly Agree NIA 
| often worked on 
activities with my 4 ( 2 ( f r 
teacher and classmates. 
| was able to find things va ( va ( rc ( 


on the class websites. 


Please rate your agreement with each of the following items after COVID-19 influenced the state of 
learning. 


Strongly 
Disagree Disagree Neutral Agree Strongly Agree NIA 


| often worked on 
activities with my @ & 4 e g C* 
teacher and classmates. 


| was able to find things = r Cc C rc oo 
on the class websites. ° . \ \ \ \ 


K-12 Survey 


Curriculum Design & Planning: Universal Design for Learning - Parents 

A quality online course adopts and implements instructional design methods that enable effective 
online instruction for both institutionally developed courses as well as licensed content from other 
sources. 


Please rate your agreement with each of the following items prior to COVID-19. 


Strongly 
Disagree Disagree Neutral Agree Strongly Agree NIA 


Class materials overall 

were easy to locate, van @ s « o a 
view, download, and use - : 
on any device. 
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Please rate your agreement with each of the following items after COVID-19 influenced the state of 


learning. 
Strongly 
Disagree Disagree Neutral Agree Strongly Agree NIA 
Class materials overall 
were easy to locate, cr ‘a cr co lan . 
‘ ( C Ci 
view, download, and use . \ > 


on any device. 


ea. tele jal oL ONLINE LEARNING 


National 
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Curriculum Design & Planning: Universal Design for Learning - Students 
A quality online course adopts and implements instructional design methods that enable effective 


online instruction for both institutionally developed courses as well as licensed content from other 
sources. 


Please rate your agreement with each of the following items prior to COVID-19. 


Strongly 
Disagree Disagree Neutral Agree Strongly Agree NIA 
My ais materials were « co co a e oC 
easy to find. - > . . ‘ Lo 
My class materials were co ~ co ~ cr = 
| i 
easy to use. “. C \ C \ C 


Please rate your agreement with each of the following items after COVID-19 influenced the state of 


learning. 
Strongly 
Disagree Disagree Neutral Agree Strongly Agree NIA 
My class materials were r* eo r~ i ry eo 
easy to find. s. 3, : a 
My class materials were ‘“ oO ‘2 a co oO 
easy to use. : 5 * \ 


amg ; i 
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Curriculum Design & Planning: Learning Objectives/Alignment with Standards - Parents 

A quality online course adopts and implements instructional design methods that enable effective 
online instruction for both institutionally developed courses as well as licensed content from other 
sources. 


Please rate your agreement with each of the following items prior to COVID-19. 


Strongly 

Disagree Disagree Neutral Agree Strongly Agree NIA 
The learning objectives 
of the classes were g ( ~ { C <= ( 


clearly stated. 


The activities in the 

classes covered the 

knowledge and skills ‘2. C ey C) = * 
needed to meet learning 

outcomes of the classes. 


Please rate your agreement with each of the following items after COVID-19 influenced the state of 
learning. 


Strongly 
Disagree Disagree Neutral Agree Strongly Agree N/A 


The learning objectives 
of the classes were & ( & C t eS 
clearly stated. 


The activities in the 

classes covered the 

knowledge and skills C (~ ~ ic Cc O 
needed to meet learning : ; ; . 
outcomes of the classes. 


ea. parecer OTN |ONLINE SEABnNG 
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Curriculum Design & Planning: Learning Objectives/Alignment with Standards - Students 

A quality online course adopts and implements instructional design methods that enable effective 
online instruction for both institutionally developed courses as well as licensed content from other 
sources. 
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Please rate your agreement with each of the following items prior to COVID-19. 


Strongly 

Disagree Disagree Neutral Agree Strongly Agree N/A 
| understood what we 
needed to learn in my £ (~ & C) r ( 
classes. 
The teachers helped me 
understand what topics 
we were going to cover ‘- (> 2 CO c~ ( 
in class and throughout 
the year. 


Please rate your agreement with each of the following items after COVID-19 influenced the state of 
learning. 
Strongly 
Disagree Disagree Neutral Agree Strongly Agree NIA 
| understood what we 
~ ~~ rm cc a ‘a 


needed to learn in my 7 ( 
classes. 


The teachers helped me 

understand what topics 

we were going to cover C ( rd (C) C : ® ] 
in class and throughout 

the year. 


| ances OL tel eeeeeae 
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Curriculum Design & Planning: Class Materials and Content - Parents 

A quality online course adopts and implements instructional design methods that enable effective 
online instruction for both institutionally developed courses as well as licensed content from other 
sources. 
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Please rate your agreement with each of the following items prior to COVID-19. 


Strongly 
Disagree Disagree Neutral Agree Strongly Agree N/A 


Class materials 

adequately covered the 

subject areas and 

included current online & C oy C & C } 
materials (online articles, 

webpages, links, and/or 

videos). 


The links to the online 

materials were secure ry a rc ry ro co 
links and avoided third ; 
party advertising. 


Online materials were oO co co a —, Co 
free to students to use. — ‘ SS \_ 3 L 


Please rate your agreement with each of the following items after COVID-19 influenced the state of 
learning. 


Strongly 
Disagree Disagree Neutral Agree Strongly Agree N/A 


Class materials 

adequately covered the 

subject areas and 

included current online @ (~ & C) e & 
materials (online articles, 

webpages, links, and/or 

videos). 


The links to the online 


materials were secure Co o o a om oO 
links and avoided third , \ : \ \ \ 
party advertising. 

Online materials were ro cC rr ‘a ry o- 
free to students to use. > : : - \ \ 


CZ, vieehsiel Coins sens 
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Curriculum Design & Planning: Class Materials and Content - Students 

A quality online course adopts and implements instructional design methods that enable effective 
online instruction for both institutionally developed courses as well as licensed content from other 
sources. 
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Please rate your agreement with each of the following items prior to COVID-19. 


Strongly 

Disagree Disagree Neutral Agree Strongly Agree N/A 
Links to the class 
materials were on the £ (~ & C) r (\ 
class website. 
Links in the class 
website all worked, so | 
could get the information ‘= ( 2 CO c~ ( 
| needed to complete my 
activities. 


Please rate your agreement with each of the following items after COVID-19 influenced the state of 
learning. 


Strongly 

Disagree Disagree Neutral Agree Strongly Agree NIA 
Links to the class 
materials were on the * & # C & (* 
class website. : 
Links in the class 
website all worked, so | 
could get the information C ( rd ( C ' ‘oe 
| needed to complete my 
activities. 


Cc ONLINE LEARNING 
CONSORTIUM 


K-12 Survey 


Online Teaching & Learning: Communication - Parents 
Online teachers support learning and facilitates presence (teacher, social, and learner) with digital 
pedagogy, communication, and engagement. 


Please rate your agreement with each of the following items prior to COVID-19. 


Strongly 
Disagree Disagree Neutral Agree Strongly Agree NIA 


Teachers hold virtual 

meeting hours and 

frequently communicate 

with students via email, co a co von co eo 
course news and ‘ : ‘ \ \ 
announcements, online 
discussions, and 
assignment feedback. 


Please rate your agreement with each of the following items after COVID-19 influenced the state of 
learning. 


Strongly 
Disagree Disagree Neutral Agree Strongly Agree NIA 


Teachers hold virtual 

meeting hours and 

frequently communicate 

with students via email, fo s co «a oo * 
course news and : : : 
announcements, online 
discussions, and 
assignment feedback. 


yiball Aysoll fT ONLINE LEARNING 
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Online Teaching & Learning: Communication - Students 
Online teachers support learning and facilitates presence (teacher, social, and learner) with digital 
pedagogy, communication, and engagement. 


Please rate your agreement with each of the following items prior to COVID-19. 


Strongly 
Disagree Disagree Neutral Agree Strongly Agree NIA 


My teacher 

communicated often 

(overall) and | was able co € co c oo « 
to contact my teacher : » 
when | needed to talk to 
them. 


My teacher sent out 

information to help me 

understand what we ‘< C ‘a co oo C 
were supposed to be 

doing. 


I talked to my r Cc co ‘a a Co 
classmates a lot online. ome ‘— s \_ = \ 


ooo 259 


\ C ONLINE LEARNING 
CONSORTIUM 


Please rate your agreement with each of the following items after COVID-19 influenced the state of 
learning. 


Strongly 
Disagree Disagree Neutral Agree Strongly Agree NIA 


My teacher 

communicated often 

(overall) and | was able vom fc co i Co la 
to contact my teacher - : . 
when | needed to talk to 
them. 


My teacher sent out 
information to help me 
understand what we 
were supposed to be 
doing. 


| talked to my -~ co cf lan i c 
classmates a lot online. a ; \ ; = 


ONLINE LEARNING 


K-12 Survey 


Online Teaching & Learning: Engagement - Parents 
Online teachers support learning and facilitates presence (teacher, social, and learner) with digital 
pedagogy, communication, and engagement. 


Please rate your agreement with each of the following items prior to COVID-19. 


Strongly 

Disagree Disagree Neutral Agree Strongly Agree NIA 
Students were 
academically ee ( r C\ oe e 
challenged. 
Students were engaged “a lan > lan e C 
with each other. : \ \ 
Students were engaged lo a cr lan _ o 
with the teacher. i : ; ‘ \ 
sShudents were engaged << co o i: ‘a co 
with the course content. — Na hs \ 
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Please rate your agreement with each of the following items after COVID-19 influenced the state of 
learning. 


Strongly 

Disagree Disagree Neutral Agree Strongly Agree NIA 
Students were 
academically & ( & (\ co {~ 
challenged. 
Students were engaged o co o oO fo oO 
with each other. ; . : aed ‘ \ 
Students were engaged r C r “ r e 
with the teacher. . 
Students were engaged Co oO Co = c Cc 


with the course content. ‘ . . * 
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Online Teaching & Learning: Engagement - Students 
Online teachers support learning and facilitates presence (teacher, social, and learner) with digital 
pedagogy, communication, and engagement. 


Please rate your agreement with each of the following items prior to COVID-19. 


Strongly 
Disagree Disagree Neutral Agree Strongly Agree N/A 
My schoolwork was hard van e rc é« rc a 


sometimes. > - ‘ . 


Please rate your agreement with each of the following items after COVID-19 influenced the state of 
learning. 


Strongly 
Disagree Disagree Neutral Agree Strongly Agree NIA 
My schoolwork was hard an of cr = oo c 


sometimes. 


PD vinall isso 
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Online Teaching & Learning: Expectation Setting - Parents 
Online teachers support learning and facilitates presence (teacher, social, and learner) with digital 
pedagogy, communication, and engagement. 


Please rate your agreement with each of the following items prior to COVID-19. 


Strongly 

Disagree Disagree Neutral Agree Strongly Agree NIA 
Expectations of students’ 
participation were clearly r ( r ] ( if 
stated online. 
Students received 
information on netiquette o> rs r> 0) ry oO 
rules and online , : : 
citizenship. 


Please rate your agreement with each of the following items after COVID-19 influenced the state of 
learning. 


Strongly 

Disagree Disagree Neutral Agree Strongly Agree N/A 
Expectations of students’ ; : 
participation were clearly C @ a C) a ( 
stated online. 
Students received 
information on netiquette ‘a o a lan oo Co 
rules and online . : ‘ . 
citizenship. 


& 
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Online Teaching & Learning: Expectation Setting - Students 
Online teachers support learning and facilitates presence (teacher, social, and learner) with digital 
pedagogy, communication, and engagement. 


Please rate your agreement with each of the following items prior to COVID-19. 


Strongly 

Disagree Disagree Neutral Agree Strongly Agree NIA 
| understood what a : es : : p 
behavior is expected of ae C & C) # ‘eo 
me online. 
| understood when my Fan C co a ~~ O 


assignments were due. 


Please rate your agreement with each of the following items after COVID-19 influenced the state of 


learning. 


| understood what 
behavior is expected of 
me online. 


| understood when my 
assignments were due. 


Strongly 
Disagree 


Disagree 
r 


C 


, vibgll 


ogi 
National clear 


Neutral 
e 
Cc 
iAjoll 
2s! BWC]: 
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Online Teaching & Learning: Outcomes - Parents 


Online teachers support learning and facilitates presence (teacher, social, and learner) with digital 


pedagogy, communication, and engagement. 


Please rate your agreement with each of the following items prior to COVID-19. 


Class learning outcomes 
were Clearly stated and 
linked with activities. 


Online courses had 
comparable outcomes to 
onsite courses. 


Strongly 
Disagree 


Disagree 


Neutral 


Agree 


Agree 


C 
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Strongly Agree 


rf 


Strongly Agree 


N/A 


0 


NIA 


Please rate your agreement with each of the following items after COVID-19 influenced the state of 


learning. 


Class learning outcomes 
were clearly stated and 
linked with activities. 


Online courses had 
comparable outcomes to 
onsite courses. 


Strongly 
Disagree 


Disagree 


Neutral 


7 


OYE CI: 


Agree 


ORT 
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tum 


Strongly Agree 


NIA 
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Online Teaching & Learning: Outcomes - Students 


Online teachers support learning and facilitates presence (teacher, social, and learner) with digital 
pedagogy, communication, and engagement. 


Please rate your agreement with each of the following items prior to COVID-19. 


Strongly 

Disagree Disagree Neutral Agree Strongly Agree NIA 
| understood what | 
needed to do in my & ( eo C) e e 


classes. 


Please rate your agreement with each of the following items after COVID-19 influenced the state of 


learning. 
Strongly 
Disagree Disagree Neutral Agree Strongly Agree NIA 
| understood what | = 
needed to do in my @ § is C- { ( 


classes. 


National 
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Online Teaching & Learning: Class Interaction - Parents 


Online teachers support learning and facilitates presence (teacher, social, and learner) with digital 
pedagogy, communication, and engagement. 


Please rate your agreement with each of the following items prior to COVID-19. 


Strongly 

Disagree Disagree Neutral Agree Strongly Agree NIA 
Student-to-student and 
teacher-to-student 
interactions were Cc € ro r = 


encouraged and ; 
facilitated in online 
courses. 
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Please rate your agreement with each of the following items after COVID-19 influenced the state of 
learning. 


Strongly 
Disagree Disagree Neutral Agree Strongly Agree NIA 


Student-to-student and 

teacher-to-student 

interactions were vom [— rc a 7 =~ 
encouraged and . 
facilitated in online 
courses, 


PDP vival jSsoll 
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Online Teaching & Learning: Class Interaction - Students 
Online teachers support learning and facilitates presence (teacher, social, and learner) with digital 
pedagogy, communication, and engagement. 


Please rate your agreement with each of the following items prior to COVID-19. 


Strongly 
Disagree Disagree Neutral Agree Strongly Agree N/A 
| worked a lot with my cr a r~ e r~ a 
classmates. ts ; ; . 
| worked i i : ‘ ‘ 
worked in groups with oO ‘a oO O i? Cc 
my classmates. * . 


Please rate your agreement with each of the following items after COVID-19 influenced the state of 
learning. 


Strongly 

Disagree Disagree Neutral Agree Strongly Agree NIA 
| worked a lot with my vom " van OO Fat Cc 
classmates. b \. th 
| worked in groups with om co o~ a van oO 
my classmates. . ; - . : 


eg, Soe BC sn 
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Online Teaching & Learning: Feedback - Parents 
Online teachers support learning and facilitates presence (teacher, social, and learner) with digital 
pedagogy, communication, and engagement. 


Please rate your agreement with each of the following items prior to COVID-19. 


Strongly 

Disagree Disagree Neutral Agree Strongly Agree NIA 
Frequent and substantial 
feedback from the co > Cc vm co —~ 
teacher was provided to . : ; : e 
students. 
Frequent and substantial 
feedback from the cr é ~ cn a vom a 
teacher was provided to ‘ ; 
parents. 


Please rate your agreement with each of the following items after COVID-19 influenced the state of 
learning. 


Strongly 
Disagree Disagree Neutral Agree Strongly Agree N/A 


Frequent and substantial 

feedback from the co — a = a e 
teacher was provided to ; ; 
students. 


Frequent and substantial 

feedback from the r fs ro 
teacher was provided to ; 
parents. 
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Online Teaching & Learning: Feedback - Students 
Online teachers support learning and facilitates presence (teacher, social, and learner) with digital 
pedagogy, communication, and engagement. 


Please rate your agreement with each of the following items prior to COVID-19. 


Strongly 
Disagree Disagree Neutral Agree Strongly Agree NIA 
The teachers told me 
-_ a rN = f a 


where | was doing well 4 { ‘ c J \ { 
and needed to improve. 
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Please rate your agreement with each of the following items after COVID-19 influenced the state of 
learning. 


Strongly 

Disagree Disagree Neutral Agree Strongly Agree NIA 
The teachers told me 
where | was doing well r C @ ¢ ‘a 2 
and needed to improve. 


yp ONLINE LEARNING 


K-12 Survey 


Assessment: Strategies - Parents 

Online teachers create and/or implement assessments in online learning environments in ways that 
ensure the validity and reliability of the instruments and procedures. Teachers measure learner 
progress through assessments, projects, and assignments that meet standards-based learning goals, 


and evaluate learner understanding of how these assessments measure achievement of the learning 
objectives. 


Please rate your agreement with each of the following items prior to COVID-19. 


Strongly 
Disagree Disagree Neutral Agree Strongly Agree NiA 


Students took quizzes 


and exams online that @ ¢ rc oe rc ¢ 
adequately captured : 

what they learned. 

Students were provided 

ample opportunity to 

show what they learned ‘- '< ee OC) « CO 


in different ways beyond 
quizzes and exams. 
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Please rate your agreement with each of the following items after COVID-19 influenced the state of 
learning. 


Strongly 
Disagree Disagree Neutral Agree Strongly Agree NIA 


Students took quizzes 

and exams online that cr co cr — on 
adequately captured ‘ ; \ : 
what they learned. 


Students were provided 
ample opportunity to 
show what they learned 
in different ways beyond 
quizzes and exams. 


MiaCl|suvears as 


K-12 Survey 


Assessment: Strategies - Students 

Online teachers create and/or implement assessments in online learning environments in ways that 
ensure the validity and reliability of the instruments and procedures. Teachers measure learner 
progress through assessments, projects, and assignments that meet standards-based learning goals, 
and evaluate learner understanding of how these assessments measure achievement of the learning 
objectives. 


Please rate your agreement with each of the following items prior to COVID-19. 


Strongly 

Disagree Disagree Neutral Agree Strongly Agree N/A 
I learned from my co ¢ rc r i 
classmates. , . ‘ \ \ \ 
| was able to show my 
teachers what | had Cj C) C oe 
learned. 
I was able to help my co vom co a f~ a 

4 \ ‘ \ L ' 


classmates learn. 
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Please rate your agreement with each of the following items after COVID-19 influenced the state of 
learning. 


Strongly 

Disagree Disagree Neutral Agree Strongly Agree NIA 
I learned from my rr ¢ Tr ry cr a 
classmates. ‘ : ; : ; 
| was able to show my 
teachers what I had C ( ss ‘a ‘2 ‘s 
learned. 
| was able to help my oe 

( C ( CO) ( C 


classmates learn. 


OL ees 
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Technology: Centralized Online Infrastructure/Operability/Security - Parents 
Educational technology is supported, reliable, accessible, and sufficient to meet the online education 
needs of teachers and students. 


Please rate your agreement with each of the following items prior to COVID-19. 


Strongly 
Disagree Disagree Neutral Agree Strongly Agree NIA 


My child was able to 

access all the things 

they needed for their s & . & c (| 
online learning in one 

place. 


My child was able to 
access the Learning 
Management System 
(LMS) with a single 
login. 


rt 
» 
a) 
") 
a 
) 


The online classes were 7 
available when my child e (~ ‘a C ® C 
needed it. 


| felt my child’s identity 
and information in the ‘e C; ge Ci G C 
class sites were secure. 
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Please rate your agreement with each of the following items after COVID-19 influenced the state of 
learning. 


Strongly 
Disagree Disagree Neutral Agree Strongly Agree NIA 


My child was able to 
access all the things 
they needed for their 
online learning in one 
place. 


My child was able to 

access the Learning 

Management System e 3 fe ‘- cj a CO 
(LMS) with a single 

login. 


The online classes were = a - 
available when my child @ C e e a e 
needed it. 


I felt my child’s identity 
and information in the C (> ‘é O C O 
class sites were secure. 


EZ, Sete MCs sees 


National 
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Technology: Centralized Online Infrastructure/Operability - Students 
Educational technology is supported, reliable, accessible, and sufficient to meet the online education 
needs of teachers and students. 


Please rate your agreement with each of the following items prior to COVID-19. 


Strongly 

Disagree Disagree Neutral Agree Strongly Agree NIA 
| was able to access all 
the things | needed for fo e co lan Fat co 
my online learning in ; : : . 3 
one place. 
| was able to access the 
Learning Management ~ ‘ ~ . om 

peel C C ‘- ‘@) ‘€ © 

System (LMS) with a 
single login. 
The online classes were - : 
available when | needed € 4 C) ( 


it 
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Please rate your agreement with each of the following items after COVID-19 influenced the state of 
learning. 


Strongly 

Disagree Disagree Neutral Agree Strongly Agree NIA 
| was able to access all 
the things | needed for rc eZ ro ea ry oO 
my online learning in > : > ; . ; 
one place. 
| was able to access the 
Learning Management [~ co co o a. oO 
System (LMS) with a - ' ? : 5 
single login. 
The online classes were r ; S f ie 
available when | needed Ey id ¢€ C @ C\ 


it 
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Technology: Security/Internet Access/Reliability - Parents/Students 
Educational technology is supported, reliable, accessible, and sufficient to meet the online education 
needs of teachers and students. 


Please rate your agreement with each of the following items prior to COVID-19. 


Strongly 

Disagree Disagree Neutral Agree Strongly Agree N/A 
Secure links were used o> ea va a rx oO 
in the class sites. b ; . 
We have internet access 
in our home that is o om o a _ la 
available when it is ° : A : : ‘ 
needed. 


The internet is 
sufficiently available in @ C . ‘@ * C * 
the school environment. 


| was consistently able 
to log in to online ‘e C o ‘ei ‘e & 


courses. 
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Please rate your agreement with each of the following items after COVID-19 influenced the state of 


learning. 

Strongly 

Disagree Disagree Neutral Agree Strongly Agree NIA 
Secure links were used r* ¢ rc rc e 
in the class sites. . \ ‘ ; . . 
We have internet access 
NOU home that iS a fo co Cc c a 
available when it is be ‘ x 
needed. 
The internet is 
sufficiently available in ¢ ( 4 () 4 ( 
the school environment. 
| was consistently able 7 : 7 : 7 
to log in to online C C 4 Cy 2 a 


courses. 
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Student Support: Student Orientation/Social-Emotional Support - Parents/Students 
Student support services are available to address the various needs of learners at different levels 
within the organization. The levels of support are appropriate and adequate for learner success. 


Please rate your agreement with each of the following items prior to COVID-19. 


Strongly 

Disagree Disagree Neutral Agree Strongly Agree N/A 
Orientation to the 
technology used in Co ea co a _ c 
online classes was * : : \ 
provided. 
Technological support CO co co c) rN Cc 

\ \ H { 1 

was available to me. . \ \ 
Support is available to 
address interpersonal la a vom a om C 
issues in online ) . . . 
courses. 
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Please rate your agreement with each of the following items after COVID-19 influenced the state of 


learning. 
Strongly 
Disagree 
Orientation to the 
technology used in cr 
online classes was > 
provided. 
Technological support co 
was available to me. , 
Support is available to 
address interpersonal - 
issues in online , 
courses. 


Disagree 


c 


Neutral Agree 
) € 
C O 
€ C 
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Strongly Agree 


Student Support: Student Orientation/Social-Emotional Support - Students 


Student support services are available to address the various needs of learners at different levels 


N/A 


within the organization. The levels of support are appropriate and adequate for learner success. 


Please rate your agreement with each of the following items prior to COVID-19. 


Strongly 
Disagree 
Academic support (such 
as library, tutoring, etc.) & 
was available to me. 
Support for my well- 
being (including co 


counseling, etc.) was 
available to me. 


| was able to make 
friends with my 
classmates and others 
online. 


-_ 


Disagree 


Neutral Agree 
# é 
i oO 
a a 


Strongly Agree 


NIA 
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Please rate your agreement with each of the following items after COVID-19 influenced the state of 
learning. 


Strongly 

Disagree Disagree Neutral Agree Strongly Agree NIA 
Academic support (such - 
as library, tutoring, etc.) @ ‘é « C r (~ 
was available to me. 
Support for my well- 
being (including co oC o lan —~ Cc 
counseling, etc.) was ios ‘ N \- 
available to me. 
| was able to make 
friends with my Co e € ea « e 
classmates and others ‘ ‘ , : 
online. 
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Student Support: Student Orientation/Social-Emotional Support - Parents 
Student support services are available to address the various needs of learners at different levels 
within the organization. The levels of support are appropriate and adequate for learner success. 


Please rate your agreement with each of the following items prior to COVID-19. 


Strongly 
Disagree Disagree Neutral Agree Strongly Agree NIA 


Legal, ethical, and safe 

behavior related to 

technology use by r~ 6 rr ro rN a 
students is supported by \ , ; \ ; 
the teacher and the 

school. 


My student was provided 

with opportunities to cr ( la i. va oO 
build relationships and ; ; , 5 ’ 
community online. 


Please rate your agreement with each of the following items after COVID-19 influenced the state of 
learning. 


Strongly 
Disagree Disagree Neutral Agree Strongly Agree NIA 


Legal, ethical, and safe 

behavior related to 

technology use by vom a eo * r e 
students is supported by ‘ : 

the teacher and the 
school. 


My student was provided 

with opportunities to rc - ct Cc) eo a7 
build relationships and ' 
community online. 
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Training & Support: Orientation - Parents 
Training and support is available for students and parents 


Please rate your agreement with each of the following items prior to COVID-19. 


Strongly 

Disagree Disagree Neutral Agree Strongly Agree NIA 
| was provided 
orientation and training 
for online courses 
(learning management @ (~ & C) « @ 
system 
access/orientation, 


guides, videos, etc.) 


Please rate your agreement with each of the following items after COVID-19 influenced the state of 
learning. 


Strongly 
Disagree Disagree Neutral Agree Strongly Agree N/A 


| was provided 

orientation and training 

for online courses 

(learning management t ( e ( C ( \ 
system 

access/orientation, 

guides, videos, etc.) 
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Evaluation & Continuous Improvement: Student Satisfaction/Class Outcomes - Parents 
Evaluation is both internal and external and informs all processes that affect teaching and learning. 
Internal evaluations often are more informal in nature and may provide immediate feedback on a 
targeted area of inquiry. External program evaluations typically look at the entire program from an 
objective perspective that will bring additional credibility to the results. 


Please rate your agreement with each of the following items prior to COVID-19. 


Strongly 

Disagree Disagree Neutral Agree Strongly Agree NIA 
Parents and students 
were provided an 
opportunity to give e co e co o a 
feedback on their . ‘ \ q 


experience with the 
online classes. 


| was informed about 

how my feedback (and 

my students’ feedback) co la co O aes oO 
was used to develop or - - . \- \ 
modify the online 

learning experience. 


Class evaluations were 

conducted to collect 

feedback on the rc va co - von - 
effectiveness of ; : 
instruction and quality of 
online class materials. 
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Please rate your agreement with each of the following items after COVID-19 influenced the state of 
learning. 


Strongly 
Disagree Disagree Neutral Agree Strongly Agree NIA 


Parents and students 

were provided an 

opportunity to give vom [— lam # r- oO 
feedback on their ‘ : ; 

experience with the 

online classes, 


| was informed about 

how my feedback (and 

my students’ feedback) Fan e" o> rt ~ en 
was used to develop or ‘ . 
modify the online 

learning experience. 


Class evaluations were 
conducted to collect 
feedback on the ry r ‘a c* ‘eo , 
effectiveness of , ; : : 
instruction and quality of 

online class materials. 
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Evaluation & Continuous Improvement: Student Satisfaction - Students 

Evaluation is both internal and external and informs all processes that affect teaching and learning. 
Internal evaluations often are more informal in nature and may provide immediate feedback ona 
targeted area of inquiry. External program evaluations typically look at the entire program from an 
objective perspective that will bring additional credibility to the results. 


Please rate your agreement with each of the following items prior to COVID-19. 


Strongly 

Disagree Disagree Neutral Agree Strongly Agree NIA 
I was given an 
opportunity to tell my 
teachers what | liked and & * & © s Ci 


didn't like about my 
online classes. 
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Please rate your agreement with each of the following items after COVID-19 influenced the state of 


learning. 
Strongly 
Disagree Disagree Neutral Agree Strongly Agree NIA 
| was given an 
opportunity to tell my 
teachers what | liked and 4 ( 4 C) 4 a 


didn't like about my 
online classes. 
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Additional Questions - Admin/Staff/Teachers 
In this section, please respond to the following questions as described. 


How often do you do let students use ICT (information and communication technology) for projects or 


classwork. 
Never or Almost Never Occasionally Frequently Always NIA 
e ® C e e 


\ 


Please indicate the extent to which you currently need professional development for ICT (information and 
communication technology) skills for teaching. 


No Need at Present Low Level of Need Moderate Level of Need High Level of Need NIA 
© * CO © O 


Was ICT (information and communication technology) skills for teaching included in your professional 
development activities during the last 12 months? 
Yes No NIA 


an ro rf 
\ \ 


In your teaching, to what extent can you support student learning through the use of digital technology (e.g. 
computers, tablets, smart boards)? 


Not at All To Some Extent Quite a Bit ALot NIA 
& @ a @ * 
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Were the following subject categories included in your formal and do you teach them during the current school 
year to any [/ 15-year-old] students in this school? CHECK ALL THAT APPLY 


Included in My Formal Education or Training | Taught It to [/ 15- year-old] Students This Year 


Information Technology 
Computer Studies 
Construction/Surveying 
Electronics 


Graphics and Design 


] 


Keyboard Skills 


1 


OOUUUOUO 


Word Processing 


Workshop/Design 
Technology 


LJ 
LJ 


The shortage or inadequacy of digital technology for instruction hindered my school's capacity to provide 
quality instruction prior to COVID. 


Not at All To Some Extent Quite a Bit ALot NIA 


Insufficient Internet access hindered my school's capacity to provide quality instruction prior to COVID. 


Not at All To Some Extent Quite a Bit ALot N/A 


School leaders took actions to support cooperation among teachers to develop new teaching practices in the 
12 months prior to the survey. 


Never Sometimes Often Very Often N/A 


Did teachers participate in a professional development network of their peers in the 12 months prior to the 
survey, 


Yes No NIA 


Thinking about the teachers in your school, how strongly do you agree or disagree that most teachers in this 
school are open to change? 


Strongly Disagree Disagree Agree Strongly Agree N/A 
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On average, how often do you take part in collaborative professional learning in your school? 
Never Once a Year or Less 2-4 Times a Year 5 Times a Year or Greater N/A 


C e ( « Cs 


\ 


During the last 12 months, did you participate in any of the following professional development activities? 


Yes No NIA 
Online ea a ‘~~ 
Courses/Seminars \/ \ ‘ 


Participation Ina 

Network of Teachers 

Formed Specifically for a fe r 
the Professional : , 
Development of 

Teachers 


To what extent do you agree with the following statements about your school’s capacity to enhance learning 
and teaching using digital devices? 


Strongly Disagree Disagree Agree Strongly Agree 


An effective online 
learning support platform * C ‘e C7 
is available 


Effective professional 

resources for teachers to of - a of 
learn how to use digital : ke 
devices are available 


Teachers have sufficient 

time to prepare lessons oo c ea ~ 
integrating digital ‘ \ 
devices 


a 
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Additional Questions - Students 
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As a student, please respond to the following questions. 


Yes No NIA 


Do you have a computer - 
at home to use for ) 
school work? 


Do you have a quiet 
place to study at home? 
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Demographic Questions - Admin/Staff/Teachers 


Sex 


Age 


20 years or younger 70 years or older 


O 


Nationality 


( ) Saudi 


() Non-Saudi 


What is the name of your school? 


What type is your school? 


( ) Governmental 
() National 


( ) International 
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Total years of experience as an educator (Please round up to whole years no matter whether you worked part- 
time or full-time and move the slider to the appropriate number of years. If any option does not apply to you 
select “O" (zero).) 


ie} 50 years or more 


Total years at your current school (Please round up to whole years no matter whether you worked part-time or 
full-time and move the slider to the appropriate number of years. If any option does not apply to you select “O" 
(zero).) 


0 50 years or more 


( 


Number of years experience with e-learning (Please round up to whole years no matter whether you worked 
part-time or full-time and move the slider to the appropriate number of years. If any option does not apply to 
you select "0" (zero).) 


0 50 years or more 
C 

Region 

(>) Riyadh 

OQ Makkah 

C Madinah 

E Asir 

(> Bahah 


Eastern Province 
Hail 
Jawt 
Jazan 
\ Northern Border 
) Najran 
\ Qassim 


(  Tabuk 
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K-12 Survey 


Teacher Specialization 


Please select your specialization(s). 


Kindergarten 


Islamic Education 


Arabic Language 


English Language 


Chinese Language 


Family Education 


Art Education 


Physical Education 


Special Education 


Social and National Education 


Computers 


Psychosocial Skills 


Mathematics 


Earth Science 


Science 


Chemistry 


Biology 


Physics 


Administration Science 


Student Counseling 


Library 


Other (please specify) 
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Demographic Questions - Parents 
Sex 
ie Male 


* Female 


Age 


20 years or younger 70 years or older 


O a 


Nationality 
> Saudi 


© Non-Saudi 


Region 
(> Riyadh 
C3 Makkah 
(> Madinah 
C™ Asir 


©™ Bahah 


© Eastern Province 


© Hail 
C™ Jawt 
© Jazan 


© Northern Border 


© Najran 
© Qassim 
© Tabuk 
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What is the name of your child/children's school? 


What type is your child/children’s school? 


( ) Governmental 
() National 


() International 


What is your highest level of schooling? 


() Secondary and below 
() High school 

( ) Diploma 

Cc Bachelors 

() Masters 


( ) Doctorate 
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Demographic Questions - Students 


Sex 


() Male 


( ) Female 


Age 


6 years or younger 18 years or older 


Nationality 
() Saudi 


( ) Non-Saudi 
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O Bahah 

() Eastern Province 
») Hail 

() Jaw 

() Jazan 

t:) Northern Border 
») Najran 

+) Qassim 


() Tabuk 
_— 


What is the name of your school? 


Po 


What type is your school? 


Cc Governmental 
() National 
—/ 


() International 


What is your grade level? 


GB? eR ole Lessee VS here esi eee 


\ KG1 


KG2 


KG3 


» Elementary 1 


Elementary 2 


‘ Elementary 3 


Elementary 4 
Elementary 5 
Elementary 6 
Middle School 1 
Middle School 2 
Middle School 3 
High School 1 
High School 2 


High School 3 
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Appendix IX: Interview Protocols 
Administrator Interview Protocol 


Interviewer: 


Post Interview Comments: 


INSTRUCTIONS 

We are conducting research for the National eLearning Center (NELC), that is intended to better understand 
the state of K-12 online learning within the Kingdom of Saudi Arabia pre-COVID-19 and during COVID-19. A 
survey has also been sent out to K-12 administrators, staff, teachers and students to obtain their 


perspectives on several dimensions of online learning. These interviews are being used to allow us to seek 
greater depth of information in specific areas. 


Interviewee Background Information 


Interviewee (Title and Name): 

School district/province: 

Does your role currently have oversight for any online learning at your school district/province? Yes / No 
If yes, please describe the extent of that oversight (e.g., school district/province-wide, 


department level, course-level, course design, etc.) 


How long have you been in your present position? 
..at this school district/province? 


What is your highest degree earned? 
What was your field of study? 


Did your school district/province offer online courses or full programs online prior to COVID? 
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Dimension |: Leadership 


Is online learning currently included in your school district/province strategic plan? 


If yes... 

la. Please describe your strategy for online learning. 

1b. Do you feel that your school district/province’s response to the pandemic was aligned to 
your online learning strategy? 


If no... 

1c. Do you anticipate adding a strategy for online learning post-COVID? Why or why not? 

1d. What are the challenges in getting the e-learning agenda going pre-COVID-19? During Covid- 
19? What is your anticipation for the future? 


Do you believe that your school district/province has sufficient resources (financial, technology, 
personnel, etc.) to support online learning? Why or why not? 


2a. How did this change due to COVID? 
2b. What resources do you believe will be needed to support online learning in the short-term? 
2c. What resources do you believe will be needed to support online learning in the long-term? 


Dimension II: Curriculum Design and Planning 


What methods did your school district/province use pre-COVID to determine course quality? 


3a. If your school district/province offered online courses, what methods were used pre-COVID 
to determine the quality of your online courses? 

3b. What changes did you make during the transition to remote learning during the Spring 2020 
term to assess the quality of your courses? 

3c. Following the COVID-19 pandemic, what methods do you think your school district/province 
will need to use to determine the quality of online courses? 


Pre-COVID, did your school district/province provide opportunities for pedagogical and technological 
innovation methods to improve student success? 


If yes... 


4a. Please share examples of how pedagogical or technological innovation methods were used 
to support student success. 
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If no... 
4b. What changes do you believe are needed at your school district/province to encourage the 
implementation of innovative pedagogical and technological methods? 


Dimension V: Technology 


Pre-COVID, did your school district/province have a technology plan? 


If yes... 

5a. Was the technology plan sufficient to support the needs of your school district/province 
during the transition to remote learning? Why or why not? 

5b. Following the COVID-19 pandemic, what changes do you believe will need to be made to it 
to better support your school district/province? 


If no... 

5c. In terms of technology, how was the shift from a physical environment to remote 
instruction? 

5d. Following the COVID-19 pandemic, does your school district/province want to develop a 
technology plan? 


5e. What do you think needs to be included in your technology plan? 


Dimension VI: Student Support 


Pre-COVID, what options were provided to students to access support services at your school 


district/province with regard to: 


a. Well-being 
b. Digital equity 
c. Learning aids 


During the transition to remote learning, what changes were made by your school district/province to 
provide students with adequate access to support services with regard to: 
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a. Well-being 
b. Digital equity 
c. Learning aids 


Following the COVID-19 pandemic, what changes does your school district/province anticipate making 
to ensure all students are able to access support services regardless of the modality (in-person, online, 
blended) of their program with regard to: 

a. Well-being 

b. Digital equity 

c. Learning aids 


Dimension VII: Training and Support 


Please describe the professional development options provided to teachers pre-COVID. 


9a. Did your school district/province provide training on how to develop quality online courses? 
9b. Did your school district/province provide training on how to teach quality online courses? 


If no to one/both... 
9c. How could the training have been improved? 


What professional development did your school district/province provide to teachers during the 
transition to remote learning? 


What changes does your school district/province anticipate making post-COVID to prepare teachers to 
teaching quality online courses? 


Dimension VIII: Evaluation of Course Outcomes and 
Program Quality 


Pre-COVID, what data did your school district/province evaluate to determine the overall quality of a 
program? 


What additional data did you review to determine quality during the transition to remote learning? 


Following the COVID-19 pandemic, what changes does your school district/province anticipate making 
in determining the process needed to assess program quality? 
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Teacher Interview Protocol 


Interviewer: 


Post Interview Comments: 


INSTRUCTIONS 

We are conducting research for the National eLearning Center (NELC), that is intended to better understand 
the state of K-12 online learning within the Kingdom of Saudi Arabia pre-COVID-19 and during COVID-19. A 
survey has also been sent out to K-12 administrators, staff, teachers, parents and students to obtain their 
perspectives on several dimensions of online learning. These interviews are being used to allow us to seek 


greater depth of information in specific areas. 


Interviewee Background Information 


Interviewee (Title and Name): 

School district/province: 

Does your role currently have oversight for any online learning at your school district/province? Yes / No 
If yes, please describe the extent of that oversight (e.g., school district/province-wide, department level, 


course-level, course design, etc.) 


How long have you been in your present position? 
..at this school district/province? 


What is your highest degree earned? 
What was your field of study? 
Did your school district/province offer online courses or full programs online prior to COVID? 


If online courses were offered, what grade levels could participate in online learning? 
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Dimension |: Leadership 


Is online learning currently included in your school district/province strategic plan? 


If yes... 

la. Please describe your strategy for online learning. 

1b. Do you feel that having a strategy for online learning improved your school 
district/province’s responsiveness during the recent pandemic? 


If no... 
1c. Do you anticipate adding a strategy for online learning post-COVID? Why or why not? 


Do you believe that your school district/province has sufficient resources (financial, technology, 
personnel, etc.) to support online learning? Why or why not? 


2a. How did this change due to COVID? 
2b. Following the COVID-19 pandemic, what resources do you believe will be needed to support 
online learning? 


Dimension II: Curriculum Design and Planning 


What methods did your school district/province use pre-COVID to determine course quality? 


3a. If your school district/province offered online courses, what methods were used pre-COVID 
to determine the quality of your online courses? 

3b. What changes did you make during the transition to remote learning to assess the quality of 
your courses? 

3c. Following the COVID-19 pandemic, what methods do you think your school district/province 
will need to use to determine the quality of online courses? 


What are your perceptions about online learning? 


Probing questions... 
4a. Do you enjoy teaching online? 
4b. Do you feel that students are able to learn in this environment? 
4c. Do you feel that this requires more or less effort by teachers teaching in this environment? 
Why? 
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Pre-COVID, did your school district/province provide opportunities for pedagogical and technological 
innovation methods to improve student success? 


If yes... 

4a. Please share examples of how pedagogical or technological innovation methods were used 
to support student success? 

If no... 


4b. What changes do you believe are needed at your school district/province to encourage the 
implementation of innovative pedagogical and technological methods? 


Dimension III: Online Teaching and Learning 


If your school district/province offered online courses pre-COVID, what methods did you use to 
encourage student engagement in the online course? 


What methods did you use during the transition to remote learning to encourage student engagement 
in their courses? 


Following the COVID-19 pandemic, what methods do you believe need to be implemented to encourage 
student engagement in their courses? 


Dimension IV: Assessment 
Pre-COVID, what methods for assessing student learning did you use in your classes? 
During the transition to remote learning, how did you assess student learning? 


Following the COVID-19 pandemic, what changes do you anticipate you will make to the methods you 
use to assess student learning? 
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Dimension VII: Training and Support 


Please describe the professional development options available to teachers pre-COVID? 


11a. Did your school district/province provide training on how to develop quality online 
courses? 


11b. Did your school district/province provide training on how to teach quality online courses? 
11c. Did your school district/province provide adequate support (subject matter experts, 
instructional designers, content developers, etc.) to help teachers design a quality online 


course? 


If no to one or more... 
11c. How could the training or support have been improved? 


What professional development was provided to you during the transition to remote learning? 


What additional training do you believe is needed post-COVID to prepare teachers to teach quality 
online courses? 
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Parent Interview Protocol 


Interviewer: 
Post Interview Comments: 
INSTRUCTIONS 


We are conducting research for the National eLearning Center (NELC), that is intended to better understand 
the state of K-12 online learning within the Kingdom of Saudi Arabia pre-COVID-19 and during COVID-19. A 


survey has also been sent out to K-12 administrators, staff, teachers, parents, and students to obtain their 
perspectives on several dimensions of online learning. These interviews are being used to allow us to seek 


greater depth of information in specific areas. 


Interviewee Background Information 


Interviewee (Title and Name): 

School district/province: 

How long has your child/have your children been enrolled in this district/province? 
What grade level is your child in/levels are your children in? 

What is your highest degree earned? 


Did your school district/province offer online courses or full programs online prior to COVID? 
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Dimension |: Leadership 


Is online learning currently included in your school district/province’s strategic plan? 


If yes... 
1a. Do you feel that your school district/province’s response to the pandemic was aligned to the 
online learning strategy? 


If no... 
1b. Following the COVID-19 pandemic, has your child/children’s school communicated about 
adding a strategy for online learning? In what ways? 


Were you aware of any challenges with online learning in your child/children’s school pre-COVID-19? 
During COVID-19? What challenges do you anticipate for the future? 


Do you believe that your school district/province has sufficient resources (financial, technology, 
personnel, etc.) to support online learning? Why or why not? 


3a. How did this change due to COVID? 


3b. What resources do you believe will be needed to support online learning in the short-term? 
3c. What resources do you believe will be needed to support online learning in the long-term? 


Dimension II: Curriculum Design & Planning 


Did your child/children participate in online learning prior to COVID-19? 


If yes... 
4a. What were your perceptions about online education prior to COVID-19? 
Ab. How did this change during COVID-19? 


If no... 


4c. What were your perceptions about online education prior to COVID-19? 
4d. How did this change during COVID-19? 
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Dimension V: Technology 


Was the technology plan of your child/children’s school(s) sufficient during the transition to remote 
learning? Why or why not? 


Following the COVID-19 pandemic, what changes do you believe will need to be made to it to better 
support your child/children’s online learning experience? 


Dimension VI: Student Support 


Pre-COVID, what options were provided to students to access support services at your school 
district/province with regard to: 

a. Well-being 

b. Digital equity 

c. Learning aids 


During the transition to remote learning, what changes were made by your school district/province to 
provide students with adequate access to support services with regard to: 

a. Well-being 

b. Digital equity 

c. Learning aids 


Following the COVID-19 pandemic, what changes should your school district/province consider to 
ensure all students are able to access support services regardless of the modality (in-person, online, 
blended) of their program with regard to: 

a. Well-being 

b. Digital equity 

c. Learning aids 


Dimension VII: Training & Support 


Pre-COVID-19, did your child/children’s school provide adequate technological training/support to you? 
To your child/children? 


During the COVID-19 pandemic, did your child/children’s school provide adequate technological 
training/support to you? To your child/children’s? 
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What needs do you anticipate with regard to technological training and support for the future? 
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National eLearning Center v, 


National eLearning Center 


https://nelc.gov.sa 


About NELC 

The National e-Learning Center was 
established as an independent 
entity by the Council of Ministers of 
Saudi Arabia, aims to enhancing 
trust in the eLearning programs, 
Leading innovation in Learning 
digital transformation and enabling 
the integration among educational 
institutions and labor market 
needs. 


ONLINE LEARNING 
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Online Learning 
Consortium 


https://onlinelearningconsortium.org/ 


About OLC 

Established in 1999 by the Sloan 
Foundation, the Online Learning 
Consortium is a collaborative 
community of higher education 
leaders and innovators, dedicated 
to advancing quality digital 
teaching and learning experiences 
designed to reach and engage the 
modern learner - anyone, anytime, 
anyplace. 
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